PROC APPEND massive log size is crashing Enterprise Guide - sas

I am using Enterprise Guide 4.2 (no choice about this).
I am using PROC APPEND to append about 80k+ observations to a SQL Server table. I believe that there are some issues with the format of the data (such as mismatching variable lengths), but it executes just fine and the table is updated. Then, Enterprise Guide gives me the following message:
The contents of Log(af771r01 (Process Flow)) is too large to display. The window will close, but the contents will remain in the project.
I was able to successfully disable the log by redirecting it to a dummy file:
** The Append Proc below outputs 80k+ lines (in theory) to the log, crashing EG 4.2;
** This statement will temporarily disable logging;
FILENAME JUNK DUMMY;
PROC PRINTTO
LOG=JUNK;
RUN;
** ========================================================
** Archive Summarized Enrollment Data
** ========================================================;
PROC APPEND BASE = Archive.MnthlyMbrCmpArch (
SASDATEFMT=(SYS_SRC_LOAD_DT='mmddyy10.')
)
DATA = Work.R1_MBR_ENRL_ARCHIVE;
RUN;
** Reenable logging;
PROC PRINTTO;
RUN;
This prevents EG from crashing, but I'm losing all of the warning and error messages. I am trying to debug the statement so I want the warnings and errors, I just don't want a flooded log.
Is there a way to partially disable logging without turning it off completely? Or is there a way to redirect the log to a file so that EG doesn't crash while trying to not open it?
I am new to SAS and am open to any suggestions, even if they do not directly answer the question.

Yes, you can redirect the log to a file by specifying a path in the FILENAME statement (instead of saying DUMMY, which refers to a non-existing file):
FILENAME JUNK 'path/file-name.log';
Of course, you must have "write" permission to the location you specify.

You can suppress a lot of information from the log by using:
options nonotes nonotes2;
And/or:
options nomprint nomlogic nosymbolgen nomacrogen; /* IF CODE CONTAINS MACROS */
Errors will still be printed to the screen but you would have little information to debug them unless you turned the logging back on.
Some SAS Documentation: http://support.sas.com/documentation/cdl/en/basess/58133/HTML/default/viewer.htm#a001906177.htm

Related

Display webout HTML generated by data steps in a SAS Web Report Studio

Here is how I build reports for SAS EnterpriseGRC:
I write sas base or macro code to collect and process data using
procs and data steps.
Then I use file _webout; in data steps and use
put statements to generate html
Then I register this code as a stored process and define filters which I am already handling in the code.
And then I navigate to Reports tab in EGRC and wala I have a report
For example my code would look like:
proc sql noprint;
CREATE TABLE work.risks AS
SELECT *
FROM opdetail.risk_L;
quit;
data _null_;
file _webout;
put '<html>';
put '<body>';
put '<table>';
put '<tr><td>Risk ID</td><td>Risk RK</td></tr>'
run;
data _null_;
set work.risks;
put '<tr>';
put '<td>'; put risk_id;'</td>'
put '<td>'; put risk_rk;'</td>'
put '</tr>';
run;
data _null_;
put '</table>';
put '</body>';
put '</html>';
run;
Now this is a very simplistic approach but a very effective one since I can theoretically represent my data in any complicated form like a correlation heat etc, as much as HTML allows
Now this technique has only one problem. Clients have a problem printing this report from within EGRC. The code I have written produces a stream output so I cant use this STP in a SAS Web Report Studio report where printing options are available. How can I solve my printing problem in the least complicated way?
p.s I know can build Informationmaps and build reports like they are supposed to but sometimes clients request demands a format of report which SAS Web Report Studio can not handle.
I could put this Print This Page in the HTML, but that puts date and page title on the top of printing page which is out of my control.
Well the least complicated way was to put a javascript print function through HTML inside the report. Though if someone IS still looking for a proper solution than ODS is the answer
Well the simplest way to print your _stream'd html report is via your trusty web browser!
Simply navigate to www.YOURSASMIDTER/SASStoredProcess and select "List Available Stored Processes and Reports". When you find your STP, right click and open in a new tab.
You now have your html report, and the URL which can be shared with end customers - who can apply your registered filters, and print your report in their browser of choice.
Of course this is not a the recommended way to build web based reports with SAS. Your SAS code will get overcomplicated by many / complex put statements (mixing SAS code with html/css/javascript). You will also struggle to apply the thousands of excellent libraries that can jazz up your report (such as highcharts, handsontable, d3).
If you are happy to deliver your reports purely from the browser (instead of EGRC) then a suggested / more scaleable approach is to use the open source SASjs adapter. More info at https://sasjs.io

Killing an entire sas process

I have developed a SAS process in Enterprise Guide 7.1 that sends e-mails daily (if need be).
The way it works is this:
[external program] generates a file which specifies who needs to be emailed and the subject matter
.
My sas process then looks like this:
1. import this file.
2. manipulate this file.
3. generate emails based on contents of manipulated file.
The problem is, everything crashes if the original file imported in step 1 is empty. Is there a way to run the import, check if the dataset is empty, and then if it is terminate the entire sas process tree?
Thank you in advance, I've been searching but to no avail.
Best way would be to put step 2 and 3 completely in a macro and only execute it when step1 dataset is not empty.
step 1 import file in dataset mydata
data _null_;
set mydata nobs=number;
call symput('mydata_count', number);
stop;
run;
%macro m;
%if &mydata_count > 0 %then %do;
step 2 manipulate this file
step 3 generate emails
%end;
%mend;
%m;
As alternative you could use the statements "Endsas" or "abort" which both terminate your job and session but they can have unwanted sideeffects, you can find these statements and information about them easily when googling for them together with keyword sas.
Although the two statements do what you originally wanted, i would recommend the logical approach i posted as first, because you have more control about what is happening that way and you can avoid some bad side-effects when working with the statements
IMO a better way is to start using a macro like %runquit;. See my answer here. https://stackoverflow.com/a/31390442/214994
Basically instead of using run; or quit; at the end of a step you use %runquit;. If any errors occurred during that step then the rest of the SAS process will be aborted. If running in batch, the entire process is killed. If running interactively, code execution stops, but your interactive session remains open.
EDIT: This assumes you get some kind of error message or warning if the file is empty.

exporting sas stored process output

I created a stored process but I want to export the output to Excel. My usual export statement doesn't work in the stored process.
%let _ODSDEST=none;
%STPBEGIN();
data x;
set sashelp.class;
run;
proc export data=x outfile = "//my documents/sp_test.xlsx" dbms=xlsx replace;
sheet="table1";
run;
* Begin EG generated code (do not edit this line);
;*';*";*/;quit;
%STPEND;
Is there a way to get this to work in the stored process?
One way to have a stored process return an excel file (actually in this case it's an xml file that excel will happily open) is to use ODS to output tagsets.excelxp (xml).
When you do this, you can use stpsrv_header to modify the HTML header. The first statement tells the browser to open the file with excel, the second tells it the file name. I believe for this header modification to work the stored process needs to deliver streaming results, not package results. But I could be wrong.
When I run below, I get a file download dialog box from the browser, allowing me to open or save the file. I'm running from Stored Process Web App, but should work fine when called from Information Delivery Portal.
%let _odsdest=tagsets.excelxp;
%let rc=%sysfunc(stpsrv_header(Content-type,application/vnd.ms-excel));
%let rc=%sysfunc(stpsrv_header(Content-disposition,attachment%str(;) filename=MyExcelFile.xls));
%stpbegin()
proc print data=sashelp.shoes (obs=&obs);
run;
%stpend()
did u check with your spelling proc exportd and outfile='mypath/my documents/myoutpt.xlsx' dbms=xlsx or outfile='mypath/my documents/myoutpt.xls' dbms=xls?? U can try with ODS also.
You can also try setting your STP up as a streaming web service, removing the %STPBEGIN and %STPEND macros, and sending to _webout using this macro: https://core.sasjs.io/mp__streamfile_8sas.html
The benefit of this, is that your code will subsequently work on Viya as well.

where data get stored in sas by using stored process. using local server(work space) option

I am using stored process for running bunch of queries and in that i am creating tables.Code runs perfectly but where the table getting stored,i am not getting it?Since in log it shows no errors. Similarly i used proc univariate, result's are displaying but where that result getting stored?I am using using local work space library to store.
As you said, your results are stored in work library. If you want to know where it is, you can see the path by running this code:
%put %sysfunc(getoption(work));
Keep in mind that work library reference is temporary and only relevant to your current session. Also, all datasets are temporary and they get wiped at the end of your SAS session.
For proc univariate, you have to specify OUT= option and point where you want your summary statistics to be saved. If the report that you are after, use ODS destinations to save it to a permanent location.
Regards,
Vasilij
To find the path being used by a libname (in this case the work libname), use this code:
%put %sysfunc(pathname(work));
For any output, it depends on your system setup, how SAS is started, etc... Your output does not necessarily go to the same folder as your work libname uses.
I would use this code in windows:
filename x pipe 'echo %cd%'; * WINDOWS COMMAND TO RETURN CURRENT WORKING DIRECTORY;
data _null_;
infile x;
input;
put _infile_;
run;
In *nix, change the filename statement to:
filename x pipe 'pwd'; * UNIX/LINUX COMMAND TO RETURN CURRENT WORKING DIRECTORY;
Or, as Vasilij suggested, use the OUT= option of the ODS statement.

clearing a library which is being used for stored compiled SAS macro

I have a program which creates a stored compiled macro in a library using the syntax:
options mstored sasmstore=MyLib;
%macro MyMac() /store source des='My Macro';
%let x=1;
%mend;
However I cannot seem to re-assign my library (MyLib) afterwards - I get the following message (sas 9.1.3):
ERROR: Unable to clear or re-assign the library MYLIB because it is still in use.
ERROR: Error in the LIBNAME statement.
Can anyone advise?
In SAS 9.3 or higher, you can now clear the libref using the %SYSMSTORECLEAR Statement.
Short answer - I don't think you can in the same SAS session.
You can't clear the libref because the macro catalog (SASMACR) remains open. It stays open since it takes resources to open and close the catalog and SAS assumes that compiled macros are going for speed in production jobs and as a trade-off, lose some dynamic abilities. All resources have to be closed before you can clear the libref. Since SASMACR remains in use (and short of closing the session doesn't appear to be a way to close it), ther is no way to clear the libref.