In SAS (9.4, if it matters) I would like to grab a CSV file from a remote host via SFTP, parse the CSV, and drop the result into a SAS data table.
I set up SFTP using PuTTY as described in the SAS docs. Binding a fileref to SFTP works okay, something like:
FILENAME mysftpfileref SFTP 'location/on/host/file.csv' HOST='myhost' USER='mysuser';
DATA _null_;
INFILE mysftpfileref TRUNCOVER;
INPUT a $25.;
RUN;
Will successfully read data.
However, I can't seem to figure out to use PROC IMPORT to actually parse the data. The docs for that proc state
"The IMPORT procedure does not support device types or access methods
for the FILENAME statement except for DISK. For example, the IMPORT
procedure does not support the TEMP device type, which creates a
temporary external file."
Is there a workaround?
You'll need to either:
Write the import code yourself (using the data step)
Download the file in some fashion and then run PROC IMPORT on the downloaded file
If you choose the second option, you can do this a few ways. The easiest is probably to write something like the above data step, read the entire line in or use the _INFILE_ automatic variable, and then write it out locally. Something along these lines (define these filenames or change them, of course):
data _null_;
infile Sftpfile;
file localf;
input #;
put _infile_;
run;
Related
I am new to SAS but used to working in python.
In python, I can read an xlsx file from a static Box.com link by calling pandas read excel function on the Box link.
pd.read_excel("https://box.com/shared/static/<url>.xlsx")
I'm hoping to do something similar in SAS. When I try
PROC IMPORT DATAFILE="https://box.com/shared/static/<url>.xlsx"
OUT=WORK.MYEXCEL
DBMS=XLSX
REPLACE;
RUN;
SAS tries to look for a document with the name "https://box.com/shared/static/.xlsx".
When I try
filename xlsxFile http "https://box.com/shared/static/<url>.xlsx";
PROC IMPORT DATAFILE=xlsxFile
OUT=WORK.MYEXCEL
DBMS=XLSX
REPLACE;
RUN;
I get
ERROR: This "filename URL" access method is not supported by "proc import". Please copy the file to local disk before running the
procedure.
Is there an easy way to have SAS access files from this type of URL? I've checked these few threads:
https://communities.sas.com/t5/General-SAS-Programming/import-excel-file-from-the-web/td-p/134158
https://communities.sas.com/t5/SAS-Programming/Importing-XLSX-from-URL-Issue/td-p/446758
https://communities.sas.com/t5/SAS-Programming/proc-import-xlsx-from-url/td-p/635834
And it seems like I may need to do some work to get SAS to create a file object from the URL but I don't quite understand how the code is working and when I naively try something similar:
filename xlsxFile http "https://box.com/shared/static/<url>.xlsx";
data file;
n=-1;
infile xlsxFile recfm=s nbyte=n length=len;
input;
file "file_name.xlsx" recfm=n;
put _infile_ $varying32767. len;
run;
PROC IMPORT OUT= input DATAFILE= "file_name.xlsx"
DBMS=xls REPLACE;
SHEET="sheet_name";
GETNAMES=YES;
RUN;
I get the following error:
Spreadsheet isn't from Excel V5 or later. Please open it in Excel and Save as V5 or later
Requested Input File Is Invalid
ERROR: Import unsuccessful. See SAS Log for details.
Any help would be appreciated. The reason I'd like to do it this way is because the files at the static links are updated daily and I want to avoid having to copy files to the SAS server every day. So if this won't work I am also interested in other work arounds that would accomplish the same thing.
I can obviously write a script that will fetch the updated files and write them to the server where SAS can access them as needed, but wanted to see if I could get this to work first.
Using the Box API from within SAS would also be another option if anyone has successfully gotten that to work. As I mentioned I am new to SAS so trying to access the API seemed like it would be too difficult at the moment.
Thank you!
You'll definitely need to download the file. As Tom notes in comments, make sure to check this is really an xlsx file; but assuming it is, some suggestions for dealing with it.
Copying it over by hand like you're doing is doable, and I'll put some code that works at the bottom of the answer, but it's way overkill nowadays - you're probably reading decade-old papers from before the modern day of SAS. In particular, you should use PROC HTTP to GET the file, rather than using the URL directly - it's just much easier that way.
Another possibility is you could use python for this! Regardless of your SAS setup, you can use python to script SAS, or use SAS to run Python, very easily nowadays. SASPy project (on github) for SAS 9, or the SAS SWAT package (also on github I believe) for connecting to SAS Viya, lets you very easily talk back and forth with Python and SAS, even in production - I do it all the time. And on top of that, you can even write SAS user-written functions in python now - so there're a bunch of ways to get your Python into SAS, if you want. I am a SAS (primary) developer, but I use Python for web-related stuff, since it's just better at it. See my paper for more details, or lots of other resources on the subject.
Assuming you just want to import the file and don't want to keep track of the excel file ultimately, you can just read it from a temp filename, which will clean itself up afterwards:
filename _httpin temp;
proc http method="get"
url="https://github.com/snoopy369/SASL/raw/master/Excel%20Precision.xlsx"
out=_httpin;
run;
proc import file=_httpin out=test dbms=xlsx replace;
run;
If you want the excel file saved somewhere (sounds like you don't, but if you do), then instead of temp assign that httpin filename to a real location.
filename _httpin "c:\temp\whatever.xlsx";
If you really want to do that binary file copy business, do it this way:
filename _httpin temp;
filename _httpout "c:\temp\Excel Precision.xlsx";
proc http method="get"
url="https://github.com/snoopy369/SASL/raw/master/Excel%20Precision.xlsx"
out=_httpin;
run;
data _null_;
_in = fopen('_httpin','I',1,'B');
_out= fopen('_httpout','O',1,'B');
rec='20'x;
do while (Fread(_in) eq 0);
rc = fget(_in,rec,1);
rc = fput(_out,rec);
rc = fwrite(_out);
end;
rc = fclose(_in);
rc = fclose(_out);
run;
proc import file=_httpout out=test dbms=xlsx replace;
run;
I've large txt file stored in sas enterprise guide(sas is connected to Winscp this is where th txt file is stored). How to read it and convert it to sas data as output.
When I check in SAS community I've get the code sample to read txt file (see bellow) is it same to read txt stored in sas?
proc import datafile='path'
out=NAME
dbms=dlm
replace;
datarow=5;
delimiter='09'x;
run;
There is another method I see also which use infile.
Which method shoul I use for me case?
I’ve not tried any method yet. Because I do not understand parameters. the path should it be the one in sas (in server) or in winscp?
Proc IMPORT works only on operating file 'references' that deliver the file directly.
WinSCP is a ftp client, so you two options:
Use WinSCP to copy the remote file to the local operating system, then you can use IMPORT or DATA step with INFILE
Use filename FTP access method and DATA step that reads data lines retrieved by SAS FTP engine
filename offsite ftp 'remote-filename` user=… pass=… host=… cd=… ;
data gotit;
infile offsite;
input var1 var2 var3 etc … ;
run;
The specific input statement might need informats and pointer control options, all dependent on the data file layout. Other infile options might be needed depending on field delimiters and content.
I have a sas7bdat format file, but it's zipped.
I could unzip the file and work on it, but this makes me lose hard disk space and time.
So I tried this code on SAS :
filename myfile ZIP 'C:\...\data.zip' member="data.sas7bdat" ;
data yoyo;
infile myfile (data.sas7bdat);
input;
put _infile_;
run;
But I get an empty yoyo table in the WORK library.
How can I successfully import the data.sas7bdat ?
Thank you,
You need to uncompress the dataset before SAS can use it. So you need to find a place that has enough space for the fully expanded file.
Note that your code is trying to specify the member name of the file within the ZIP file twice. You should only do that once. Either point the fileref to the aggregate location and use member name in the reference. Or point the fileref to the individual member and just use the fileref.
Here is a method to expand the file into your current WORK folder.
%let member=data.sas7bdat;
filename in zip 'C:\...\data.zip' member="&member" recfm=n;
filename out "%sysfunc(pathname(work))/&member" recfm=n;
data _null_;
rc=fcopy('in','out');
run;
You can now work with the file using the name WORK.DATA.
proc print data=work.data(obs=1); run;
If you want to read data from a ZIP file directly then it either needs to be raw (text) data or in a streaming format, like a SAS V5 XPORT file.
I created a stored process but I want to export the output to Excel. My usual export statement doesn't work in the stored process.
%let _ODSDEST=none;
%STPBEGIN();
data x;
set sashelp.class;
run;
proc export data=x outfile = "//my documents/sp_test.xlsx" dbms=xlsx replace;
sheet="table1";
run;
* Begin EG generated code (do not edit this line);
;*';*";*/;quit;
%STPEND;
Is there a way to get this to work in the stored process?
One way to have a stored process return an excel file (actually in this case it's an xml file that excel will happily open) is to use ODS to output tagsets.excelxp (xml).
When you do this, you can use stpsrv_header to modify the HTML header. The first statement tells the browser to open the file with excel, the second tells it the file name. I believe for this header modification to work the stored process needs to deliver streaming results, not package results. But I could be wrong.
When I run below, I get a file download dialog box from the browser, allowing me to open or save the file. I'm running from Stored Process Web App, but should work fine when called from Information Delivery Portal.
%let _odsdest=tagsets.excelxp;
%let rc=%sysfunc(stpsrv_header(Content-type,application/vnd.ms-excel));
%let rc=%sysfunc(stpsrv_header(Content-disposition,attachment%str(;) filename=MyExcelFile.xls));
%stpbegin()
proc print data=sashelp.shoes (obs=&obs);
run;
%stpend()
did u check with your spelling proc exportd and outfile='mypath/my documents/myoutpt.xlsx' dbms=xlsx or outfile='mypath/my documents/myoutpt.xls' dbms=xls?? U can try with ODS also.
You can also try setting your STP up as a streaming web service, removing the %STPBEGIN and %STPEND macros, and sending to _webout using this macro: https://core.sasjs.io/mp__streamfile_8sas.html
The benefit of this, is that your code will subsequently work on Viya as well.
I need to import data from European Social Survey databank to SAS.
I'm not very good at using SAS so I just naively tried importing the text file one gets but it stores it all in one variable.
Can someone maybe help me with what to do? Since there doesn't seem to be a guide on their webpage I reckon it has to be pretty easy.
It's free to register (and takes 5 secs) and I need all possible data for Denmark.
Edit: When downloading what they call a SAS file, what i get is a huge proc format and the same text file as one gets by choosing text.
The data in the text file isn't comma separated and the first row does not contain variable names.
Download it in SAS format. Save the text file in a location you can remember, and open the SAS file. It's not just one big proc format; it's a big proc format followed by a datastep with input code. It was probably created by SPSS (it fits the pattern of an SPSS saved .sas file anyhow). Look for:
DATA OUT.ESS1_4e01_0_F1;
Or something like that (that's what it is when I downloaded it). It's probably about 3/4 of the way down the page. You just need to change the code:
INFILE 'ESS1_4e01_0_F1.txt';
or similar, to be the directory you placed the text file in. Create a LIBNAME for OUT that goes to wherever you want to permanently save this, and do that at the start of the .sas file, replacing the top 3 lines like so.
Originally:
LIBNAME LIBRARY '';
LIBNAME OUT '';
PROC FORMAT LIBRARY=LIBRARY ;
Change these to:
libname out "c:\mystuff\"; *but probably not c:\mystuff :);
options fmtsearch=(out);
proc format lib=out;
Then run the entire thing.
This is the best solution if you want the formatted values (value labels) and variable labels. If you don't care about that, then it might be easier to deal with the CSV like Bob shows.
But the website says yu can download SAS format, why don't you?
You need a delimiter if all goes into one column.
data temp;
length ...;
infile 'file.csv' dlm=',';
input ...;
run;
As Dirk says, the web site says you can download a SAS dataset directly. However, if there's some reason you don't want to do that, choose a comma separated file (CSV) and use PROC IMPORT. Here is an example:
proc import out=some_data
datafile='c:\path\somedata.csv'
dbms=csv replace;
getnames=yes;
run;
Of course, this assumes the first row contains column names.