I was running the following code:
%LET TIME_INTERVAL='MINUTE15';
/*
* Get the file names of a specific location
*/
%MACRO get_filenames(location,filenames);
filename _dir_ "%bquote(&location.)";
data &filenames(keep=fname);
handle=dopen( '_dir_' );
if handle > 0 then do;
count=dnum(handle);
do i=1 to count;
fname=dread(handle,i);
output &filenames;
end;
end;
rc=dclose(handle);
run;
filename _dir_ clear;
%MEND;
%MACRO NBBO (fname);
DATA TICKERS_NBBO;
INFILE &fname;
INPUT SYMBOL $;
RUN;
%mend;
%MACRO CALCU(DATE_VAR);
%get_filenames('./groups',filenames);
data _null_;
set filenames;
by fname;
if fname =: "&TIME_INTERVAL";
%NBBO(fname);
run;
%mend;
However, I got the error: ERROR: No logical assign for filename FNAME.
I was wondering what is the reason that caused this?
There are lots of csv files in the folder groups. I was trying to run the NBBO macro on each of the files and load each of the files into a dataset with infile statement.
You're mixing up data step code and macro code in a manner that's not permitted. You can't extract the contents of the variable fname and use it to populate a macro variable in this way. Instead you're passing the text fname to that macro variable, which then doesn't work.
You also can't run another data step inside a data step in this manner. You need to use one of the various methods for executing it after the data step terminates or during execution.
Finally, it's entirely unclear to me what you're really doing here: because you're going to end up with just one of those files in the dataset.
I think you may want something like this, which is much easier. I use the FILEVAR option in infile, which uses the value of a variable; I have to make some changes to how you calculate fname (I think you have to do this anyway, it has no directory name in it).
%MACRO get_filenames(location,filenames);
filename _dir_ "%bquote(&location.)";
data &filenames(keep=fname);
length fname $512;
handle=dopen( '_dir_' );
if handle > 0 then do;
count=dnum(handle);
do i=1 to count;
fname=catx('\',"%bquote(&location.)",dread(handle,i));
output &filenames;
end;
end;
rc=dclose(handle);
run;
filename _dir_ clear;
%MEND;
%get_filenames(location=c:\temp\cars, filenames=fnam);
data tickers_nbbo;
set fnam;
infile a filevar=fname dlm=',';
input symbol $;
run;
If you really do need to call a data step separately, you either need to use CALL EXECUTE, DOSUBL, or construct macro calls in another way (PROC SQL SELECT INTO, %INCLUDE, etc.).
Related
I wanna append string to a text file at beginning and end of proc sql statment, I tried like below
libname DXZ 'libpath';
%macro processlogger(msg);
filename logfile '../Processlog/processlog.txt';
data _null_;
file logfile;
put "%superq(message)";
run;
%mend;
%processlogger ('Begin');
proc sql;
select * from DZ.NoofDaysin_Reje /* Mispelled name */
run;
%processlogger('End');
I seems to messing up in macro variable, is there any other way I can do this, Thanks
If you want to use a data step to append to a text file then you need to add the MOD keyword to the FILE statement.
If you want to print the value of a macro variable that might have quotes and other strange characters in a data step then it is probably best to use symget() to retrieve the value into a datastep variable and print that.
Make sure to reference the macro variable that you created msg and not some other macro variable message.
If you don't want quotes to be included in the value of a macro variable then do not add them.
%macro processlogger(msg);
data _null_;
file '../Processlog/processlog.txt' mod;
length message $32767 ;
message=symget('msg');
put message ;
run;
%mend;
%processlogger(Starting at %sysfunc(datetime(),datetime24.3));
%processlogger(Ending at %sysfunc(datetime(),datetime24.3));
You can also use PRINTTO to redirect your log to a text file. And you have the option to either append to the original file or replace. Example here.
/*Redirect your log file*/
proc printto log='../Processlog/processlog.txt';
run;
/* Your Code Here */
/* Reset log path to default */
proc printto;
run;
I was trying to get all of the fname variables with the prefix TIME_INTERVAL. However, it seems that the following put statement outputs nothing. I was wondering what is the reason?
%LET TIME_INTERVAL='MINUTE15';
data _null_;
set filenames;
by fname;
if fname =: "&TIME_INTERVAL";
put fname;
run;
If I put the put statement above the if statement, however, it can output correctly:
%LET TIME_INTERVAL='MINUTE15';
data _null_;
set filenames;
by fname;
put fname;
if fname =: "&TIME_INTERVAL";
run;
Update:
The content of the dataset filenames:
MINUTE15_group0.csv
MINUTE15_group1.csv
MINUTE15_group2.csv
MINUTE15_group3.csv
MINUTE15_group4.csv
MINUTE15_group5.csv
MINUTE15_group6.csv
MINUTE15_group7.csv
MINUTE15_group8.csv
MINUTE5_group0.csv
MINUTE5_group1.csv
MINUTE5_group2.csv
MINUTE5_group3.csv
MINUTE5_group4.csv
MINUTE5_group5.csv
MINUTE5_group6.csv
MINUTE5_group7.csv
MINUTE5_group8.csv
SECOND5_group0.csv
SECOND5_group1.csv
SECOND5_group2.csv
SECOND5_group3.csv
SECOND5_group4.csv
SECOND5_group5.csv
SECOND5_group6.csv
SECOND5_group7.csv
SECOND5_group8.csv
You asked SAS to look for filenames that start with 'MINUTE15', but I think that you actually wanted to look for filenames that start with MINUTE15 instead.
Everything is a string to the macro processor so there is no need to add quotes around constant text in macro code. If you add them they become part of the code that is being generated and passed to SAS to run. So your program generated this IF statement.
if fname =: "'MINUTE15'";
You could remove the single quotes.
%LET TIME_INTERVAL=MINUTE15;
data _null_;
set filenames;
if fname =: "&TIME_INTERVAL";
put fname;
run;
Or remove the double quotes.
%LET TIME_INTERVAL='MINUTE15';
data _null_;
set filenames;
if fname =: &TIME_INTERVAL;
put fname;
run;
i m new to sas and studying different ways to do subject line task.
Here is two ways i knew at the moment
Method1: file statement in data step
*DATA _NULL_ / FILE / PUT ;
data _null_;
set engappeal;
file 'C:\Users\1502911\Desktop\exportdata.txt' dlm=',';
put id $ name $ semester scoreEng;
run;
Method2: Proc Export
proc export
data = engappeal
outfile = 'C:\Users\1502911\Desktop\exportdata2.txt'
dbms = dlm;
delimiter = ',';
run;
Question:
1, Is there any alternative way to export raw data files
2, Is it possible to export the header also using the data step method 1
You can also make use of ODS
ods listing file="C:\Users\1502911\Desktop\exportdata3.txt";
proc print data=engappeal noobs;
run;
ods listing close;
You need to use the DSD option on the FILE statement to make sure that delimiters are properly quoted and missing values are not represented by spaces. Make sure you set your record length long enough, including delimiters and inserted quotes. Don't worry about setting it too long as the lines are variable length.
You can use CALL VNEXT to find and output the names. The LINK statement is so the loop is later in the data step to prevent __NAME__ from being included in the (_ALL_) variable list.
data _null_;
set sashelp.class ;
file 'class.csv' dsd dlm=',' lrecl=1000000 ;
if _n_ eq 1 then link names;
put (_all_) (:);
return;
names:
length __name__ $32;
do while(1);
call vnext(__name__);
if upcase(__name__) eq '__NAME__' then leave;
put __name__ #;
end;
put;
return;
run;
Dataset: Have
F1 F2
Student Section
Name No
Dataset "Have". Data has new line character.
I need to compress the newline character from the data.
I want to do this dynamically as sometimes the "Have" dataset may contain new variables like F3,F4,F5 etc.,
I have written as macro to do this.. However it is not working as expected.
When i execute the below code, first time I am getting error as invalid reference newcnt. If i execute for second time in the same session, i am not getting error.
PFB my code:
%macro update_2(newcnt);
data HAVE;
set HAVE;
%do i= 1 %to &newcnt;
%let colname = F&i;
&colname=compress(&colname,,'c');
%end;
run;
%mend update_2;
%macro update_1();
proc sql noprint;
select count(*) into :cnt from dictionary.columns where libname="WORK" and memname="HAVE";
quit;
%update_2(&cnt)
%mend update_1;
Note: All the variables have name as F1,F2,F3,F4.,
Please tell me what is going wrong..
If there is any other procedures, please help me.
In your macro %update_1 you're creating a macro variable called &cnt, but when you call %update_2 you refer to another macro variable, &colcnt. Try fixing this reference and see if your code behaves as expected.
We created our own function to clean unwanted characters from strings using proc fcmp. In this case, our function cleans tab characters, line feeds, and carriage returns.
proc fcmp outlib=common.funcs.funcs; /* REPLACE TARGET DESTINATION AS NECESSARY */
function clean(iField $) $200;
length cleaned $200;
bad_char_list = byte(10) || byte(9) || byte(13);
cleaned = translate(iField," ",bad_char_list);
return (cleaned );
endsub;
run;
Create some test data with a new line character in the middle of it, then export it and view the results. You can see the string has been split across lines:
data x;
length employer $200;
employer = cats("blah",byte(10),"diblah");
run;
proc export data=x outfile="%sysfunc(pathname(work))\x.csv" dbms=csv replace;
run;
Run our newly created clean() function against the string and export it again. You can see it is now on a single line as desired:
data y;
set x;
employer = clean(employer);
run;
proc export data=y outfile="%sysfunc(pathname(work))\y.csv" dbms=csv replace;
run;
Now to apply this method to all character variables in our desired dataset. No need for macros, just define an array referencing all the character variables, and iterate over them applying the clean() function as we go:
data cleaned;
set x;
array a[*] _char_;
do cnt=lbound(a) to hbound(a);
a[cnt] = clean(a[cnt]);
end;
run;
EDIT : Also note that fcmp may have some performance considerations to consider. If you are working with very large amounts of data, there may be other solutions that will perform better.
EDIT 6/15/2020 : Corrected missing length statement that could result in truncated responses.
Here's an example of Robert Penridge's function, as a call routine with an array as an argument. This probably only works in 9.4+ or possibly later updates of 9.3, when permanent arrays began being allowed to be used as arguments in this way.
I'm not sure if this could be done flexibly with an array as a function; without using macros (which require recompilation of the function constantly) I don't know how one could make the right size of array be returned without doing it as a call routine.
I added 'Z' to the drop list so it's obvious that it works.
options cmplib=work.funcs;
proc fcmp outlib=work.funcs.funcs;
sub clean(iField[*] $);
outargs iField;
bad_char_list = byte(11)|| byte(10) || byte(9) || byte(13)||"Z";
do _i = 1 to dim(iField);
iField[_i] = translate(iField[_i],trimn(" "),bad_char_list);
end;
endsub;
quit;
data y;
length employer1-employer5 $20;
array employer[4] $;
do _i = 1 to dim(employer);
employer[_i] = "Hello"||byte(32)||"Z"||"Goodbye";
end;
employer5 = "Hello"||byte(32)||"Z"||"Goodbye";
call clean(employer);
run;
proc print data=y;
run;
Here is another alternative. If newline is the only thing you want to remove, then we are talking about Char only, you may leverage implicit array and Do over,
data want;
set have;
array chr _character_;
do over chr;
chr=compress(chr,,'c');
end;
run;
Here is the macro code.....
libname myfmt "&FBRMrootPath./Formats";
%macro CreateFormat(DSN,Label,Start,fmtname,type);
options mprint mlogic symbolgen;
%If &type='n' %then %do;
proc sort data=&DSN out=Out; by &Label;
Run;
Data ctrl;
set Out(rename=(&Label=label &Start=start )) end=last;
retain fmtname &fmtname type &type;
output;
If last then do;
hlo='O';
label='*ERROR';
output;
End;
Run;
%End;
%Else %do;
proc sort data=&DSN out=Out; by &Start;
Run;
Data ctrl;
set Out(rename=(&Start=label &Label=start )) end=last;
retain fmtname &fmtname type &type;
output;
If last then do;
hlo='O';
label='*ERROR';
output;
End;
Run;
%End;
proc format library=myfmt cntlin=ctrl;
Run;
%Mend CreateFormat;
Here is the code for control data set through which above macro should run for each observation of the data set and the values of the observations are inputs for varibales in the macro....
Data OPER.format_control;
Input DSN :$12. Label :$15. Start :$15. fmtName :$8. type :$1. fmt_Startdt :mmddyy. fmt_Enddt :mmddyy.;
format fmt_Startdt fmt_Enddt date9.;
Datalines;
ssin.prd prd_nm prd_id mealnm n . 12/31/9999
ssin.prd prd_id prd_nm mealid c . 12/31/9999
ssin.fac fac_nm onesrc_fac_id fac1SRnm n . 12/31/9999
ssin.fac fac_nm D3_fac_id facD3nm n . 12/31/9999
ssin.fac onesrc_fac_id D3_fac_id facD31SR n . 12/31/9999
oper.wrkgrp wrkgrp_nm wrkgrp_id grpnm n . 12/31/9999
;
Something like this.
proc sql;
select catx(',',cats('%CreateFormat(',DSN),Label,Start,fmtname,cats(type,')');
into :formcreatelist separated by ' '
from oper.format_control;
quit;
You may need to PUT some of your variables to get the format you want into the macro variable. I use the slightly cludgy cats/catx combo here, you could cats once with ',' added in a bunch of times also.
You do have a limit here - around 20,000 characters total in a macro variable. If it's over that, you either have to use CALL EXECUTE (which has some quirky features) or you can put the macro call into a text file and %INCLUDE it.
There is a better way to do this rather than select ... into a macro variable. Use a temp file like this:
filename dyncode temp;
data _null_;
file dyncode;
set OPER.format_control;
put '%createformat ....';
run;
%include dyncode;
filename dyncode clear;
This technique is not limited by the 32k length limitation on macro variables.
Note that you should definitely use single quotes around the %createformat to prevent SAS from invoking the macro just prior to data step compilation. You want the macro to run when the %include runs.
The above approach is analogous to call execute, but call execute is evil because it does not execute the macro and embedded data/proc code within the macro in the expected order. Avoid call execute.
Finally, if you are running interactive SAS and using the technique there is a neat trick you can use to debug. Comment out the last two lines of code -- the include and the filename clear. After you run the remaining code, enter the SAS command "fslist dyncode" in the command window. This will pop up a notepad view on the dynamic code you just generated. You can review it and make sure you got what you intended.
Here's a call execute solution, just for completeness:
data _null_;
set OPER.format_control;
call execute('%CreateFormat(' || DSN || ',' || Label || ',' || Start || ',' || fmtname || ',' || type || ');');
run;