my task in SAS is to upload a file via SFTP with extension tmp and after the upload is finished to rename it to csv. As I am no adminstrator of the server my debugging output is limited and I am struggeling with the correct implementation. The following code generates no error in the SAS Log but does not rename the file:
%let host=...;
%let sftpUser = ...;
%let filename_tmp=20160301-test01-sas.tmp;
%let filename=20160301-test01-sas.csv;
%let sftpPath=...;
FILENAME test SFTP "&sftpPath.&filename_tmp."
HOST="&host."
USER="&sftpUser."
DEBUG;
proc export data=.... outfile=test dbms=csv replace;
run;
data _null_;
rc=rename("test&sftpPath.&filename_tmp.", "test&sftpPath.&filename.", 'file');
run;
I already read the docs for filename and rename but I could not get a clue on how to combine both statements - any help or hints or alternatives are greatly appreciated.
Thanks
Stephan
There is a script running on the destination server scanning for csv files and move them every minute so there is the danger that a file is moved when the upload is not completed.
I'm pretty sure that there's no danger of the file being moved, as it should be locked until the upload completes. Just try it.
Related
I am new to SAS but used to working in python.
In python, I can read an xlsx file from a static Box.com link by calling pandas read excel function on the Box link.
pd.read_excel("https://box.com/shared/static/<url>.xlsx")
I'm hoping to do something similar in SAS. When I try
PROC IMPORT DATAFILE="https://box.com/shared/static/<url>.xlsx"
OUT=WORK.MYEXCEL
DBMS=XLSX
REPLACE;
RUN;
SAS tries to look for a document with the name "https://box.com/shared/static/.xlsx".
When I try
filename xlsxFile http "https://box.com/shared/static/<url>.xlsx";
PROC IMPORT DATAFILE=xlsxFile
OUT=WORK.MYEXCEL
DBMS=XLSX
REPLACE;
RUN;
I get
ERROR: This "filename URL" access method is not supported by "proc import". Please copy the file to local disk before running the
procedure.
Is there an easy way to have SAS access files from this type of URL? I've checked these few threads:
https://communities.sas.com/t5/General-SAS-Programming/import-excel-file-from-the-web/td-p/134158
https://communities.sas.com/t5/SAS-Programming/Importing-XLSX-from-URL-Issue/td-p/446758
https://communities.sas.com/t5/SAS-Programming/proc-import-xlsx-from-url/td-p/635834
And it seems like I may need to do some work to get SAS to create a file object from the URL but I don't quite understand how the code is working and when I naively try something similar:
filename xlsxFile http "https://box.com/shared/static/<url>.xlsx";
data file;
n=-1;
infile xlsxFile recfm=s nbyte=n length=len;
input;
file "file_name.xlsx" recfm=n;
put _infile_ $varying32767. len;
run;
PROC IMPORT OUT= input DATAFILE= "file_name.xlsx"
DBMS=xls REPLACE;
SHEET="sheet_name";
GETNAMES=YES;
RUN;
I get the following error:
Spreadsheet isn't from Excel V5 or later. Please open it in Excel and Save as V5 or later
Requested Input File Is Invalid
ERROR: Import unsuccessful. See SAS Log for details.
Any help would be appreciated. The reason I'd like to do it this way is because the files at the static links are updated daily and I want to avoid having to copy files to the SAS server every day. So if this won't work I am also interested in other work arounds that would accomplish the same thing.
I can obviously write a script that will fetch the updated files and write them to the server where SAS can access them as needed, but wanted to see if I could get this to work first.
Using the Box API from within SAS would also be another option if anyone has successfully gotten that to work. As I mentioned I am new to SAS so trying to access the API seemed like it would be too difficult at the moment.
Thank you!
You'll definitely need to download the file. As Tom notes in comments, make sure to check this is really an xlsx file; but assuming it is, some suggestions for dealing with it.
Copying it over by hand like you're doing is doable, and I'll put some code that works at the bottom of the answer, but it's way overkill nowadays - you're probably reading decade-old papers from before the modern day of SAS. In particular, you should use PROC HTTP to GET the file, rather than using the URL directly - it's just much easier that way.
Another possibility is you could use python for this! Regardless of your SAS setup, you can use python to script SAS, or use SAS to run Python, very easily nowadays. SASPy project (on github) for SAS 9, or the SAS SWAT package (also on github I believe) for connecting to SAS Viya, lets you very easily talk back and forth with Python and SAS, even in production - I do it all the time. And on top of that, you can even write SAS user-written functions in python now - so there're a bunch of ways to get your Python into SAS, if you want. I am a SAS (primary) developer, but I use Python for web-related stuff, since it's just better at it. See my paper for more details, or lots of other resources on the subject.
Assuming you just want to import the file and don't want to keep track of the excel file ultimately, you can just read it from a temp filename, which will clean itself up afterwards:
filename _httpin temp;
proc http method="get"
url="https://github.com/snoopy369/SASL/raw/master/Excel%20Precision.xlsx"
out=_httpin;
run;
proc import file=_httpin out=test dbms=xlsx replace;
run;
If you want the excel file saved somewhere (sounds like you don't, but if you do), then instead of temp assign that httpin filename to a real location.
filename _httpin "c:\temp\whatever.xlsx";
If you really want to do that binary file copy business, do it this way:
filename _httpin temp;
filename _httpout "c:\temp\Excel Precision.xlsx";
proc http method="get"
url="https://github.com/snoopy369/SASL/raw/master/Excel%20Precision.xlsx"
out=_httpin;
run;
data _null_;
_in = fopen('_httpin','I',1,'B');
_out= fopen('_httpout','O',1,'B');
rec='20'x;
do while (Fread(_in) eq 0);
rc = fget(_in,rec,1);
rc = fput(_out,rec);
rc = fwrite(_out);
end;
rc = fclose(_in);
rc = fclose(_out);
run;
proc import file=_httpout out=test dbms=xlsx replace;
run;
I created a stored process but I want to export the output to Excel. My usual export statement doesn't work in the stored process.
%let _ODSDEST=none;
%STPBEGIN();
data x;
set sashelp.class;
run;
proc export data=x outfile = "//my documents/sp_test.xlsx" dbms=xlsx replace;
sheet="table1";
run;
* Begin EG generated code (do not edit this line);
;*';*";*/;quit;
%STPEND;
Is there a way to get this to work in the stored process?
One way to have a stored process return an excel file (actually in this case it's an xml file that excel will happily open) is to use ODS to output tagsets.excelxp (xml).
When you do this, you can use stpsrv_header to modify the HTML header. The first statement tells the browser to open the file with excel, the second tells it the file name. I believe for this header modification to work the stored process needs to deliver streaming results, not package results. But I could be wrong.
When I run below, I get a file download dialog box from the browser, allowing me to open or save the file. I'm running from Stored Process Web App, but should work fine when called from Information Delivery Portal.
%let _odsdest=tagsets.excelxp;
%let rc=%sysfunc(stpsrv_header(Content-type,application/vnd.ms-excel));
%let rc=%sysfunc(stpsrv_header(Content-disposition,attachment%str(;) filename=MyExcelFile.xls));
%stpbegin()
proc print data=sashelp.shoes (obs=&obs);
run;
%stpend()
did u check with your spelling proc exportd and outfile='mypath/my documents/myoutpt.xlsx' dbms=xlsx or outfile='mypath/my documents/myoutpt.xls' dbms=xls?? U can try with ODS also.
You can also try setting your STP up as a streaming web service, removing the %STPBEGIN and %STPEND macros, and sending to _webout using this macro: https://core.sasjs.io/mp__streamfile_8sas.html
The benefit of this, is that your code will subsequently work on Viya as well.
I'm trying to use a Macro that retrieves a single value from a CSV file. I've written a MACRO that works perfectly fine if there is only 1 CSV file, but does not deliver the expected results when I have to run it against more than one file. If there is more than one file it returns the value of the last file in each iteration.
%macro reporting_import( full_file_route );
%PUT The Source file route is: &full_file_route;
%PUT ##############################################################;
PROC IMPORT datafile = "&full_file_route"
out = file_indicator_tmp
dbms = csv
replace;
datarow = 3;
RUN;
data file_indicator_tmp (KEEP= lbl);
set file_indicator_tmp;
if _N_ = 1;
lbl = "_410 - ACCOUNTS"n;
run;
proc sql noprint ;
select lbl
into :file_indicator
from file_indicator_tmp;
quit;
%PUT The Source Reporting period states: &file_indicator;
%PUT ##############################################################;
%mend;
This is where I execute the Macro. Each excel file's full route exists as a seperate record in a dataset called "HELPERS.RAW_WAITLIST".
data _NULL_;
set HELPERS.RAW_WAITLIST;
call execute('%reporting_import('||filename||')');
run;
In the one example I just ran, The one file contains 01-JUN-2015 and the other 02-JUN-2015. But what the code returns in the LOG file is:
The Source file route is: <route...>\FOO1.csv
##############################################################
The Source Reporting period states: Reporting Date:02-JUN-2015
##############################################################
The Source file route is: <route...>\FOO2.csv
##############################################################
The Source Reporting period states: Reporting Date:02-JUN-2015
##############################################################
Does anybody understand why this is happening? Or is there perhaps a better way to solve this?
UPDATE:
If I remove the code from the MACRO and run it manually for each input file, It works perfectly. So it must have something to do with the MACRO overwriting values.
CALL EXECUTE has tricky timing issues. When it invokes a macro, if that macro generates macro variables from data set variables, it's a good idea to wrap the macro call in %NRSTR(). That way call execute generates the macro call, but doesn't actually execute the macro. So try changing your call execute statement to:
call execute('%nrstr(%%)reporting_import('||filename||')');
I posted a much longer explanation here.
I'm not too clear on the connections between your files. But instead of importing the CSV files and then searching for your string, couldn't you use a pipe command to save the results of a grep search on your CSV files to a dataset and then read just in the results?
Update:
I tried replicating your issue locally and it works for me if I set file_indicator with a call symput as below instead of your into :file_indicator:
data file_indicator_tmp (KEEP= lbl);
set file_indicator_tmp;
if _N_ = 1;
lbl = "_410 - ACCOUNTS"n;
data _null_ ;
set file_indicator_tmp ;
if _n_=1 then call symput('file_indicator',lbl) ;
run;
I am using SAS Enterprise Guide 6.1 and am utilizing the "Submit SAS Code when server is connected" capability under Tools, SAS Programs to submit all of my personal credentials, libnames. Yes, I know there is probably an easier way to do this with SAS Management Console, but that is not an option right now.
I have several Teradata libraries I need to assign within each project, but the problem is that sometimes I (or more commonly somebody else on my team) will change my password and forget to change it in the start-up code. This results in several incorrect attempts and locks me out immediately. I wish to do a few things:
Create a conditional libname assignment that will execute all the libnames if the credentials are correct.
If the credentials are incorrect, the libnames are not executed (so that I don't lock myself out).
Since the SAS code in the "Submit SAS Code when server is connected" section doesn't seem to generate a log, I wish to send myself an email with the SAS Log attached (only if the credentials fail).
Kill the Server connection (if credentials error out) to avoid further attempts to assign libraries.
Here is my attempt, I'm having trouble attaching the log and setting up the email statement.
*Define personal credentials;
%let myemail=gollum#middleearth.com;
filename temp email "&myemail";
*Define Teradata credentials;
%let tera_user=Gollum; /*Teradata Username*/
%let tera_pwd=#filthy_hobbitses; /*Teradata Password*/
*Conditionally assign libraries;
%macro libsetup();
libname library1 teradata user=&tera_user password="&tera_pwd" tdpid=terap schema=library1 fastload=yes bulkload=yes fastexport=yes;
%if &syslibrc=0 %then
%do;
libname library2 teradata user=&tera_user password="&tera_pwd" tdpid=terap schema=library2 fastload=yes bulkload=yes fastexport=yes;
libname library3 teradata user=&tera_user password="&tera_pwd" tdpid=terap schema=library3 fastload=yes bulkload=yes fastexport=yes;
*more library statements here;
%end;
%else
%do;
data _null_;
file temp
subject="TERADATA CREDENTIALS ERROR"
attach=("put SAS LOG filename here");
put 'Teradata Login Failed. SAS LOG Attached.';
%abort abend;
%end;
%mend libsetup;
%libsetup;
Thanks.
Can't tell if your looking for suggestions on alternative approaches, or for help with the emailing.
For emailing, something like below (untested) should work:
filename __mymail email
to="gollum#middleearth.com"
from="gollum#middleearth.com"
subject="credential error"
attach="/home/mylog.log"
;
data _null_;
file __mymail;
put "Hi!";
run;
Note the server where SAS is executing has to have access to a mail server, and actual code needed may vary with mail protocol, etc.
For attaching the log, will probably need to use PROC PRINTTO to write your log to a file, and then use PROC PRINTTO again to let go of the file before you email it.
HTH
I wanted to export SAS dataset from SAS into FTP. I can export csv file (or txt file) using the following command:
%macro export_to_ftp(dsn= ,outfile_name= );
Filename MyFTP ftp "&outfile_name."
HOST='ftp.site.com'
cd= "&DATA_STRM/QC"
USER=&ftp_user.
PASS=&ftp_pass.;
PROC EXPORT DATA= &dsn. OUTFILE= MyFTP DBMS=%SCAN(&outfile_name.,2,.) REPLACE;
RUN; filename MyFTP clear;
%mend;
%export_to_ftp(dsn=lib1.dataset ,outfile_name=dataset.csv);
But couldn't use it to export SAS dataset. Can anybody please help me.
Thank you!
PROC EXPORT is not used to export SAS datasets, it's used to convert SAS datasets to other formats. You normally wouldn't use the FTP filename method to transfer SAS datasets; you would either use SAS/CONNECT if you are intending to transfer from one SAS machine to another (if you license SAS/CONNECT and want help with this, please say so), or use normal (OS) FTP processes to transfer the file. It is technically possible to use the FTP filename method to transfer a SAS file (as a binary file, reading then writing byte-by-byte) but that's error-prone and overly complicated.
The best method if you're using SAS to drive the process is to write a FTP script in your OS, and call that using x or %sysmd, passing the filename as an argument. If you include information about your operating system, something could be easily drawn up to help you out.
Note: if you're on a server, you need to verify that you have 'x' permission; that's often locked down. If you do not, you may not be able to run this entirely from SAS.
As Joe says, you do not use PROC EXPORT to create a file to be transferred using FTP. The safest way to exchange SAS datasets is to use PROC CPORT to create a transport file. Here is a modified version of your original macro:
%macro export_to_ftp(dsn= ,outfile_name= );
%let DBMS=%UPCASE(%SCAN(&outfile_name.,2,.));
%if &DBMS ne CSV and &DBMS ne TXT and &DBMS ne CPT %then %do;
%put &DBMS is not supported.;
%goto getout;
%end;
%if &DBMS=CPT %then %do;
filename MyFTP ftp "&outfile_name."
HOST='ftp.site.com'
cd= "&DATA_STRM/QC"
USER=&ftp_user.
PASS=&ftp_pass.
rcmd='binary';
PROC CPORT DATA= &dsn.
FILE = MyFTP;
RUN;
%end;
%else %do;
filename MyFTP ftp "&outfile_name."
HOST='ftp.site.com'
cd= "&DATA_STRM/QC"
USER=&ftp_user.
PASS=&ftp_pass.
rcmd='ascii';
PROC EXPORT DATA= &dsn.
OUTFILE= MyFTP
DBMS= &dbms REPLACE;
RUN;
%end;
filename MyFTP clear;
%getout:
%mend;
%export_to_ftp(dsn=lib1.dataset ,outfile_name=dataset.csv);
%export_to_ftp(dsn=lib1.dataset ,outfile_name=dataset.cpt);
By convention, this will use a file extension of cpt to identify that you want a SAS transport file created. Whoever receives the file would use PROC CIMPORT to convert the file back to a SAS dataset:
filename xpt 'path-to-transport-file';
proc cimport data=dataset infile=xpt;
run;
filename xpt clear;
Note that SAS transport files should be transferred as binary files; the other two formats are text files; hence the different filename statements.
One of many advantages of using PROC CPORT is that the entire data set is copied, including any indexes that may exist. Also, you are protected against problems related to using the data set on operating systems different from the one that created it.
What OP requires is a Binary transfer to the FTP server. That can be done via a data step.
filename ftpput ftp "<full name of your file with ext>" cd='<DIR>' user='<username>' pass='<password>' host='<ftp host>' recfm=s debug; /*ftp dir stream connection*/
filename myfile '/path/to/your/library/dsfile.sas7bdat' recfm=n; /*local file*/
/*Binary Transfer -- recfm=n*/
data _null_;
n=1;
infile myfile nbyte=n;
input;
file ftpput;
put _infile_ ##;
run;
You can tweak the input/put statements to your specifications. But otherwise, this works for me.
I guess, you can also try the fcopy function with the above filenames. That should work too, with binary transfers.
options nonotes; /*do not want the data step or ftp server notes*/
data _null_;
fcop=fcopy("myfile","ftpput");
if fcop = 0 /*Success code*/ then do;
put '|Successfully copied src file to FTP!|';
end;
else do;
msg=sysmsg();
put fcop= msg=;
end;
run;
options notes;
Looking at your code, you seem to have forgotten the quotes (") around &ftp_user. and &ftp_pass.
Otherwise your code looks okay to me.
If that does not do the trick, some error message would come in handy.
Also note that your use of scan to determine the dbms is tricky: what if a future filename has (multiple) dots in it? you are better of putting -1 (part after last dot) instead of 2 (part after second dot) as a parameter to your scan function.
I would actually do it the other way around
Export your file locally and then upload it with the FTP program
something like this: (note I used CSV.. but please use what ever file format)
You may need to edit that a little more but the base logic is there
%macro export_to_ftp(dsn= ,outfile_name= );
PROC EXPORT DATA= &dsn. OUTFILE= &file_to_FTP..CSV DBMS=%SCAN(&outfile_name.,2,.) REPLACE;
RUN;
Filename MyFTP ftp "c:\FTP_command.bat"
put "ftp &ftp_user.:&ftp_pass.#ftp.site.com "
put "cd &DATA_STRM./QC"
put "c:\&file_to_FTP..CSV ";
x "c:\FTP_command.bat";
%mend;
%export_to_ftp(dsn=lib1.dataset ,outfile_name=dataset.csv);