I am creating a dataset using filename URL web submissions. However, in some instaces I keep getting '502' responses from the server. To get around this I would like to use some conditional logic inside a macro. I'm most of the way there, but I cant quite get the end bit to work. The idea is that the macro, which is nested within other nested macros will keep trying this one submission until it gets a dataset that doesn't have 0 observations then move on:
%macro test_exst;
filename loader url "http://finance.yahoo.com/d/quotes.csv?s=&svar1.+&svar2.+&svar3.+&svar4.+&svar5.+&svar6.+&svar7.+&svar8.+&svar9.+&svar10.+
&svar11.+&svar12.+&svar13.+&svar14.+&svar15.+&svar16.+&svar17.+&svar18.+&svar19.+&svar20.+
&svar21.+&svar22.+&svar23.+&svar24.+&svar25.+&svar26.+&svar27.+&svar28.+&svar29.+&svar30.+
&svar31.+&svar32.+&svar33.+&svar34.+&svar35.+&svar36.+&svar37.+&svar38.+&svar39.+&svar40.+
&svar41.+&svar42.+&svar43.+&svar44.+&svar45.+&svar46.+&svar47.+&svar48.+&svar49.+&svar50.+
&svar51.+&svar52.+&svar53.+&svar54.+&svar55.+&svar56.+&svar57.+&svar58.+&svar59.+&svar60.+
&svar61.+&svar62.+&svar63.+&svar64.+&svar65.+&svar66.+&svar67.+&svar68.+&svar69.+&svar70.+
&svar71.+&svar72.+&svar73.+&svar74.+&svar75.+&svar76.+&svar77.+&svar78.+&svar79.+&svar80.+
&svar81.+&svar82.+&svar83.+&svar84.+&svar85.+&svar86.+&svar87.+&svar88.+&svar89.+&svar90.+
&svar91.+&svar92.+&svar93.+&svar94.+&svar95.+&svar96.+&svar97.+&svar98.+&svar99.+&svar100.+
&svar101.+&svar102.+&svar103.+&svar104.+&svar105.+&svar106.+&svar107.+&svar108.+&svar109.+&svar110.+
&svar111.+&svar112.+&svar113.+&svar114.+&svar115.+&svar116.+&svar117.+&svar118.+&svar119.+&svar120.+
&svar121.+&svar122.+&svar123.+&svar124.+&svar125.+&svar126.+&svar127.+&svar128.+&svar129.+&svar130.+
&svar131.+&svar132.+&svar133.+&svar134.+&svar135.+&svar136.+&svar137.+&svar138.+&svar139.+&svar140.+
&svar141.+&svar142.+&svar143.+&svar144.+&svar145.+&svar146.+&svar147.+&svar148.+&svar149.+&svar150.+
&svar151.+&svar152.+&svar153.+&svar154.+&svar155.+&svar156.+&svar157.+&svar158.+&svar159.+&svar160.+
&svar161.+&svar162.+&svar163.+&svar164.+&svar165.+&svar166.+&svar167.+&svar168.+&svar169.+&svar170.+
&svar171.+&svar172.+&svar173.+&svar174.+&svar175.+&svar176.+&svar177.+&svar178.+&svar179.+&svar180.+
&svar181.+&svar182.+&svar183.+&svar184.+&svar185.+&svar186.+&svar187.+&svar188.+&svar189.+&svar190.+
&svar191.+&svar192.+&svar193.+&svar194.+&svar195.+&svar196.+&svar197.+&svar198.+&svar199.+&svar200.
&f=&&fvar&a." DEBUG ;
/* data step based on filename url above goes here, each pass will give 500 metrics x 1 symbol dataset*/
%put create dataset from csv submission;
data temp_&I._&&fvar&a.;
infile loader length=len MISSOVER /*delimiter = ','*/;
/* input record $varying8192. len; */
input record $varying30. len;
format record $30.;
informat record $30.;
run;
data _null_;
dsid=open("temp_&I._&&fvar&a.");
obs=attrn(dsid,"nobs");
put "number of observations = " obs;
if obs = 0 then stop;
else;
filename loader url "http://finance.yahoo.com/d/quotes.csv?s=&svar1.+&svar2.+&svar3.+&svar4.+&svar5.+&svar6.+&svar7.+&svar8.+&svar9.+&svar10.+
&svar11.+&svar12.+&svar13.+&svar14.+&svar15.+&svar16.+&svar17.+&svar18.+&svar19.+&svar20.+
&svar21.+&svar22.+&svar23.+&svar24.+&svar25.+&svar26.+&svar27.+&svar28.+&svar29.+&svar30.+
&svar31.+&svar32.+&svar33.+&svar34.+&svar35.+&svar36.+&svar37.+&svar38.+&svar39.+&svar40.+
&svar41.+&svar42.+&svar43.+&svar44.+&svar45.+&svar46.+&svar47.+&svar48.+&svar49.+&svar50.+
&svar51.+&svar52.+&svar53.+&svar54.+&svar55.+&svar56.+&svar57.+&svar58.+&svar59.+&svar60.+
&svar61.+&svar62.+&svar63.+&svar64.+&svar65.+&svar66.+&svar67.+&svar68.+&svar69.+&svar70.+
&svar71.+&svar72.+&svar73.+&svar74.+&svar75.+&svar76.+&svar77.+&svar78.+&svar79.+&svar80.+
&svar81.+&svar82.+&svar83.+&svar84.+&svar85.+&svar86.+&svar87.+&svar88.+&svar89.+&svar90.+
&svar91.+&svar92.+&svar93.+&svar94.+&svar95.+&svar96.+&svar97.+&svar98.+&svar99.+&svar100.+
&svar101.+&svar102.+&svar103.+&svar104.+&svar105.+&svar106.+&svar107.+&svar108.+&svar109.+&svar110.+
&svar111.+&svar112.+&svar113.+&svar114.+&svar115.+&svar116.+&svar117.+&svar118.+&svar119.+&svar120.+
&svar121.+&svar122.+&svar123.+&svar124.+&svar125.+&svar126.+&svar127.+&svar128.+&svar129.+&svar130.+
&svar131.+&svar132.+&svar133.+&svar134.+&svar135.+&svar136.+&svar137.+&svar138.+&svar139.+&svar140.+
&svar141.+&svar142.+&svar143.+&svar144.+&svar145.+&svar146.+&svar147.+&svar148.+&svar149.+&svar150.+
&svar151.+&svar152.+&svar153.+&svar154.+&svar155.+&svar156.+&svar157.+&svar158.+&svar159.+&svar160.+
&svar161.+&svar162.+&svar163.+&svar164.+&svar165.+&svar166.+&svar167.+&svar168.+&svar169.+&svar170.+
&svar171.+&svar172.+&svar173.+&svar174.+&svar175.+&svar176.+&svar177.+&svar178.+&svar179.+&svar180.+
&svar181.+&svar182.+&svar183.+&svar184.+&svar185.+&svar186.+&svar187.+&svar188.+&svar189.+&svar190.+
&svar191.+&svar192.+&svar193.+&svar194.+&svar195.+&svar196.+&svar197.+&svar198.+&svar199.+&svar200.
&f=&&fvar&a." DEBUG ;
data temp_&I._&&fvar&a.;
infile loader length=len MISSOVER /*delimiter = ','*/;
/* input record $varying8192. len; */
input record $varying30. len;
format record $30.;
informat record $30.;
run;
run;
%mend;
%test_exst;
The idea here is try URL submission, create dataset from it, check number of obs is not zero. If its not end the macro. If it is resubmit the same filename URL then create the dataset from it again. Keep doing this until the server respond, the exit the macro and most on to the rest of the code.
I haven't got as far as running this code in anger yet. I'm guessing the filename URL will work fine, but it is the fact that the code is attempting to create a dataset within a data null step right at the end that is making it fall over. Any ideas?
Thanks
Without getting into the specifics of your project, a good way to approach this generally is with recursion.
%macro test_me(iter);
%let iter=%eval(&iter.+1);
data my_data;
infile myfilename;
input stuff;
call symputx("obscount",_n_);
run;
%if &obscount=1 and &iter. < 10 %then %do;
%put Iteration &iter. failed, trying again;
%test_me(&iter.);
%end;
%mend test_me;
%test_me(0);
It checks to see if it worked, and if it did not work, it calls itself again, with a maximum iteration count to make sure you don't end up in infinite loop land if the server is down or somesuch. You also might put a delay in there if the server has a maximum frequency of calls or any other rules the API requires you to follow.
Related
I was running the following code:
%LET TIME_INTERVAL='MINUTE15';
/*
* Get the file names of a specific location
*/
%MACRO get_filenames(location,filenames);
filename _dir_ "%bquote(&location.)";
data &filenames(keep=fname);
handle=dopen( '_dir_' );
if handle > 0 then do;
count=dnum(handle);
do i=1 to count;
fname=dread(handle,i);
output &filenames;
end;
end;
rc=dclose(handle);
run;
filename _dir_ clear;
%MEND;
%MACRO NBBO (fname);
DATA TICKERS_NBBO;
INFILE &fname;
INPUT SYMBOL $;
RUN;
%mend;
%MACRO CALCU(DATE_VAR);
%get_filenames('./groups',filenames);
data _null_;
set filenames;
by fname;
if fname =: "&TIME_INTERVAL";
%NBBO(fname);
run;
%mend;
However, I got the error: ERROR: No logical assign for filename FNAME.
I was wondering what is the reason that caused this?
There are lots of csv files in the folder groups. I was trying to run the NBBO macro on each of the files and load each of the files into a dataset with infile statement.
You're mixing up data step code and macro code in a manner that's not permitted. You can't extract the contents of the variable fname and use it to populate a macro variable in this way. Instead you're passing the text fname to that macro variable, which then doesn't work.
You also can't run another data step inside a data step in this manner. You need to use one of the various methods for executing it after the data step terminates or during execution.
Finally, it's entirely unclear to me what you're really doing here: because you're going to end up with just one of those files in the dataset.
I think you may want something like this, which is much easier. I use the FILEVAR option in infile, which uses the value of a variable; I have to make some changes to how you calculate fname (I think you have to do this anyway, it has no directory name in it).
%MACRO get_filenames(location,filenames);
filename _dir_ "%bquote(&location.)";
data &filenames(keep=fname);
length fname $512;
handle=dopen( '_dir_' );
if handle > 0 then do;
count=dnum(handle);
do i=1 to count;
fname=catx('\',"%bquote(&location.)",dread(handle,i));
output &filenames;
end;
end;
rc=dclose(handle);
run;
filename _dir_ clear;
%MEND;
%get_filenames(location=c:\temp\cars, filenames=fnam);
data tickers_nbbo;
set fnam;
infile a filevar=fname dlm=',';
input symbol $;
run;
If you really do need to call a data step separately, you either need to use CALL EXECUTE, DOSUBL, or construct macro calls in another way (PROC SQL SELECT INTO, %INCLUDE, etc.).
Although I have the error handling:
filename myfile email
to=&e_mail.
subject= "Error: &number."
type="text/plain";
%macro errmail;
%if &syserr ne 0 %then %do;
options obs= max replace nosyntaxcheck;
data _null_;
file myfile;
put;
put 'ERROR';
put "&syserrortext";
put 'check a log'
run;
%abort cancel;
%end;
%mend errmail;
when I have the error in the proc export:
(&number. is the table in the work)
proc export data=&number.
outfile="/usr/local/backup/&number./&number._%SYSFUNC(TODAY(),DATE9.).txt"
replace
dbms=dlm;
delimiter=';';
run;
%errmail;
ERROR: Physical file does not exist, /usr/local/backup/2116/2116_13MAY2016.txt.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 1 observations read from the data set WORK.2116.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.01 seconds
SAS dosen't check a macro and doesn't send an e-mail. I have an error in the path, so I understand the error, but I need an e-mail about that and I want that SAS stops processing because of macro.
Adding some options to the macro can fix it?
this is how I send out my email from SAS
not required to create a file on the system
filename outbox email "email#address.com";
data _null_;
file outbox
to=("email#address.com")
cc=("email#address.com")
subject="SUBJECT OF EMAIL "
/* attach=("C:\sas\results.out" "C:\sas\code.sas") */ ; /* incase you need to attach a file */
put 'Hello! ';
put ' ';
put 'Thank you & let me know if there is anything,';
put "¯ovar."; /*in case you need to put some detail in the email from macro variables make sure to use double quote for that */
put 'SIGNATURE ME';
run;
With the below code the email always gets sent. 1 obviously does not equal 0, but yet it still runs. I have tried removing the do part but still get the same issue.
data _null_;
set TestTable;
if 1 = 0 then do;
file sendit email
to=("email#gmail.com")
subject="Some Subject Line";
end;
run;
While the file statement is considered an executable statement (and thus should not be executed when behind a false if statement), that is not really entirely true. SAS sees the file statement during compilation, and knows that it needs to create a file to write to - and thus, it's somewhat compile-time. That's what is happening here - SAS creates the file (in this case, the email) as a result of the compiler's activity, then doesn't actually populate it with anything, but still has an email at the end of the day.
The same thing happens with any other file - like so:
data _null_;
set sashelp.class;
if 0 then do;
file "c:\temp\test_non_zero.txt";
put name $;
end;
run;
A blank file is created by that code.
If you need to conditionally send emails, I would recommend wrapping your email code in a macro, and then calling that macro using call execute or similar from the dataset. Like so:
%macro write_email(parameters);
data _null_;
file sendit email
to=("email#gmail.com")
subject="Some Subject Line";
run;
%mend write_email;
data _null_;
set TestTable;
if 0 then do;
call execute('%write_email(parameters)');
end;
run;
Use email directives to abort the message.
http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a002058232.htm
data _null_;
file sendit email to=("email#gmail.com") subject="Some Subject Line";
if nobs=0 then put '!EM_ABORT!';
set TestTable nobs=nobs;
....
run;
I am an intermediate user of SAS, but I have limited knowledge of arrays and macros. I have a set of code that prompts the user to enter a date range. For example, the user might enter December 1, 2015-December 5,2015. For simplicity, imagine the code looks like:
data new; set old;
if x1='December 1, 2015'd then y="TRUE";
run;
I need to run this same code for every day in the date prompt range, so for the 1st, 2nd, 3rd, 4th, and 5th. My thought was to create an array that contains the dates, but I am not sure how I would do that. My second thought was to create a macro, but I can't figure out out to feed a list through a macro.
Also, just FYI, the code is a lot longer and more complicated than just a data step.
The following macro can be used as a framework for your code:
%MACRO test(startDate, endDAte);
%DO i=&startDate %to &endate;
/* data steps go here */
/* example */
DATA test;
SET table;
IF x1 = &i THEN y = "true";
RUN;
%END;
%MEND;
Look into call execute to call your macro and a data null step using a do loop to loop through the days. Getting the string correct for the call execute can sometimes be tricky, but worth the effort overall.
data sample;
do date='01Jan2014'd to '31Jan2014'd;
output;
end;
run;
%macro print_date(date);
proc print data=sample;
where date="&date"d;
format date date9.;
run;
%mend;
%let date_start=05Jan2014;
%let date_end=11Jan2014;
data _null_;
do date1="&date_start"d to "&date_end"d by 1;
str='%print_date('||put(date1, date9.)||');';
call execute(str);
end;
run;
I have a dataset called have with one entry with multiple variables that look like this:
message reference time qty price
x 101 35000 100 .
the above dataset changes every time in a loop where message can be ="A". If the message="X" then this means to remove 100 qty from the MASTER set where the reference number equals the reference number in the MASTER database. The price=. is because it is already in the MASTER database under reference=101. The MASTER database aggregates all the available orders at some price with quantity available. If in the next loop message="A" then the have dataset would look like this:
message reference time qty price
A 102 35010 150 500
then this mean to add a new reference number to the MASTER database. In other words, to append the line to the MASTER.
I have the following code in my loop to update the quantity in my MASTER database when there is a message X:
data b.master;
modify b.master have(where=(message="X")) updatemode=nomissingcheck;
by order_reference_number;
if _iorc_ = %sysrc(_SOK) then do;
replace;
end;
else if _iorc_ = %sysrc(_DSENMR) then do;
output;
_error_ = 0;
end;
else if _iorc_ = %sysrc(_DSEMTR) then do;
_error_ = 0;
end;
else if _iorc_ = %sysrc(_DSENOM) then do;
_error_ = 0;
end;
run;
I use the replace to update the quantity. But since my entry for price=. when message is X, the above code sets the price='.' where reference=101 in the MASTER via the replace statement...which I don't want. Hence, I prefer to delete the price column is message=X in the have dataset. But I don't want to delete column price when message=A since I use this code
proc append base=MASTER data=have(where=(msg_type="A")) force;
run;
Hence, I have this code price to my Modify statement:
data have(drop=price_alt);
set have; if message="X" then do;
output;end;
else do; /*I WANT TO MAKE NO CHANGE*/
end;run;
but it doesn't do what I want. If the message is not equal X then I don't want to drop the column. If it is equal X, I want to drop the column. How can I adapt the code above to make it work?
Its a bit of a strange request to be honest, such that it raises questions about whether what you're doing is the best way of doing it. However, in the spirit of answering the question...
The answer by DomPazz gives the option of splitting the data into two possible sets, but if you want code down the line to always refer to a specific data set, this creates its own complications.
You also can't, in the one data step, tell SAS to output to the "same" data set where one instance has a column and one instance doesn't. So what you'd like, therefor, is for the code itself to be dynamic, so that the data step that exists is either one that does drop the column, or one that does not drop the column, depending on whether message=x. The answer to this, dynamic code, like many things in SAS, resolves to the creative use of macros. And it looks something like this:
/* Just making your input data set */
data have;
message='x';
time=35000;
qty=1000;
price=10.05;
price_alt=10.6;
run;
/* Writing the macro */
%macro solution;
%local id rc1 rc2;
%let id=%sysfunc(open(work.have));
%syscall set(id);
%let rc1=%sysfunc(fetchobs(&id, 1));
%let rc2=%sysfunc(close(&id));
%IF &message=x %THEN %DO;
data have(drop=price_alt);
set have;
run;
%END;
%ELSE %DO;
data have;
set have;
run;
%END;
%mend solution;
/* Running the macro */
%solution;
Try this:
data outX(drop=price_alt) outNoX;
set have;
if message = "X" then
output outX;
else
output outNoX;
run;
As #sasfrog says in the comments, a table either has a column or it does not. If you want to subset things where MESSAGE="X" then you can use something like this to create 2 data sets.