I was wondering if I could libname a directory containing serveral folders with datasets I need.
The directory looks like this:
Directory_A
--Directory_B
----Dataset_1.sas7bdat
--Directory_C
----Dataset_2.sas7bdat
----Dataset_3.sas7bdat
--Directory_D
----Dataset_4.sas7bdat
Cheers.
You cannot point a library at a parent folder and automatically display all the subfolder contents, however you can configure a "concatenated libname" with each of the individual subfolders like so:
libname yourlib ('Directory_B' 'Directory_C' 'Directory D');
You can also mix and match with existing librefs, eg:
libname yourlib ('Directory_B' librefX 'Directory C');
for more info, see documentation.
Related
I am relatively new to sas. I have a data set in one of my folders among the Explorer but I need to have it in my Library called WORK so that I can use it, the file ends with .sas7bdat
How can I do that?
You don't need it in your WORK to work with it.
To use MY_TABLE.sas7bdat in c:\my\path, you just declare a library libname MY_LIB 'c:\my\path';, which you then use (more or less) as a tables-space or Db2 schema. For instance
data MY_DATA; set MY_LIB.MY_TABLE;
select ... from MY_LIB.MY_TABLE where ...;
I am using BIG QUERY EXPORT DATA statement to create files in cloud storage for an another team to extract for further reprocessing. I am using below statement, not pasting the select query as its huge.
EXPORT DATA OPTIONS(
uri='gs://whr-asia-datalake-dev-standard/outbound/Adobe/Customer_Master_*.csv',
format='CSV',
overwrite=true,
header=true,
field_delimiter='|') AS
SELECT
I see below files getting created in my cloud storage bucket
radhika_sharma_ibm#cloudshell:~ (whr-asia-datalake-nonprod)$ gsutil ls gs://whr-asia-datalake-dev-standard/outbound/Adobe/
gs://whr-asia-datalake-dev-standard/outbound/Adobe/
gs://whr-asia-datalake-dev-standard/outbound/Adobe/Customer_Master_000000000000.csv
gs://whr-asia-datalake-dev-standard/outbound/Adobe/Customer_Master_000000000001.csv
gs://whr-asia-datalake-dev-standard/outbound/Adobe/Customer_Master_000000000002.csv
I cannot remove the suffix part as BIG QUERY creates it, but I am wondering if I can create files with DATE in the file name for the other team to identify what date it is created for??
That is like
Customer_Master_04022021_000000000000_.csv
I need to have a date in my file. Any help or inputs please?
Is there a work around or I will have to go with a data flow here that is using a data flow job to extract data from table in a file.
You can use the uri value as:
'gs://bucket/folder/your_filename-'||current_datetime()||'-*.csv'
Either Current_date() or current_datetime() can be used.
Thanks
I just got started in PowerBI and I am generating two report files every month from Service NOW.
SLA's report and the Incident report. Eventually, these files have the naming INC_MM_YY.xls or SLA_MM_YY.xls.
I am trying to make the addition of the previous month's files without the need to add new data sources/edit the queries. It seems that it is possible using M language in the advanced query editor but seems a lot complicated since I have 0 experience with power query M.
Are there other ways?
Or in the case above. I can retrieve the folder data as a table and iterate over the files. But how to do that in the M language?
Thank you.
EDIT: Just to try to make it clear let's look at the table generated by the folder source.
We have the name of the file and it's path for each row.
So in pseudo code should be something like:
For (each row as n) {
if (n.folderpath ends with "sla") {
tablesla += load source n."folderpath" && n."filename"
}
else tableincident += load source n."folderpath" && n."filename"
}
It just seems not practical in powerquery :/ I could find how to make something similar to a for loop but very confusing.
I figured it out.
You can actually create two different sources, one for the folder with the SLA and another with the folder for incident. Just after combining and transforming the data from one of the folders. Still in the Query Editor, you just click New Source and the other folder data will combined in a different table.
With that you have two distinct tables and any time when you put a new file in one of the folders, hit refresh, the data will be added to the correct table.
Thank you guys.
try the load from folder option, you can place each months files into a its own folder one for the SLA's and one for the Incidents. With the load from folder, it will go though each file and load it. So the next month, you add in Novembers data, refresh the dataset(s) and it will add it automatically.
The files need to be the same structure for it to work effectively, and it will load what it sees in the folder, so if you remove a file, Power BI will not retain it in the workbook, it only loads what it can see.
Other examples
https://powerbi.tips/2016/06/loading-data-from-folder/
https://insightsoftware.com/blog/power-bi-load-data-from-folder/
Hope that helps
I've used this code to fetch the list of objects for all SAS Libraries, Physical Tables and Jobs.
https://github.com/sasjs/core/blob/master/meta/mm_getobjects.sas
I now need to fetch these objects details,
Like for Libraries - I need their libname and full path,
Teradata Libs - Schema Name,Lib path
Physical Tables - Location and other attribs
Jobs - Location, and other attribs.
I'm not very familiar on how or what attribs can we report, but I definitely need their paths and attribs.
Thank you.
The example you refer to is using proc metadata that returns XML you need to understand and process. The real problem here is you have to learn how to build input XML to construct the metadata query, which is quite a complex thing.
Maybe more straight forward is to use Data step metadata functions like here.
METABROWSE command is useful to understand metadata object relations (if you have access to SAS Foundation), see here
The attributes you are asking for will change depending on the library engine you are checking for.
The following macro will generate a libname for BASE, OLEDB, ODBC and POSTGRES engines (note that the repository has moved):
https://github.com/sasjs/core/blob/main/meta/mm_assigndirectlib.sas
The direct attributes are available as per this answer: How to get details of metadata objects in SAS
Folder path is available as per this answer:
%let metauri=OMSOBJ:PhysicalTable\A5HOSDWY.BE0006N9;
/* get metadata paths */
data ;
length tree_path $500 tree_uri parent_uri parent_name $200;
call missing(tree_path,tree_uri,parent_uri,parent_name);
drop tree_uri parent_uri parent_name rc ;
uri="&metauri";
rc=metadata_getnasn(uri,"Trees",1,tree_uri);
rc=metadata_getattr(tree_uri,"Name",tree_path);
do while (metadata_getnasn(tree_uri,"ParentTree",1,parent_uri)>0);
rc=metadata_getattr(parent_uri,"Name",parent_name);
tree_path=strip(parent_name)||'/'||strip(tree_path);
tree_uri=parent_uri;
end;
tree_path='/'||strip(tree_path);
run;
I have a template for a report in a folder on our server, and instead of copying the template into a new directory every time I need to run a report, I'm looking automate it in SAS.
I have tried using x command and system() as below:
data _null_;
rc = system('copy "\\servername\Reports\Reporting Templates\Template.xlsm" "\\servername\Output\Template_new.xlsm")
put rc =;
run;
but I run into the error ERROR: Shell escape is not valid in this SAS session., which appears to be security related and I need admin rights to activate the option.
Then from google searching, I came across https://www.sas.com/content/dam/SAS/en_ca/User%20Group%20Presentations/TASS/Jia_Lin_Manage_External_Files_June2015.pdf
which mentions using the rename() function within a SAS data step to rename an external file (and copy to a new folder, the author claims). However, I cannot get it to work.
data _null_;
rc = rename("\\servername\Reports\Reporting Templates\Template.xlsm",
"\\servername\Reports\Reporting Templates\Template_new.xlsm",
"file");
put rc=;
run;
When I check the log, I have no errors and rc has a value of 1 which, according to the documentation, means that it was executed successfully, but the file name has not changed.
I have also tried this in different folders and ondifferent file types to no avail. What am I doing wrong?
It's possible your SAS session is locked down and unable to modify files except in a few locations. From the SAS documentation on RENAME()
If the SAS session in which you are specifying the FILEEXIST function
is in a locked-down state, and the pathname specified in the function
has not been added to the lockdown path list, then the function will
fail and a file access error related to the locked-down data will not
be generated in the SAS log unless you specify the SYSMSG function.
Good job, SAS Documentation team for not changing FILEEXIST to RENAME during your copy and paste.
Use the SYSMSG() function to see if there is an error that is not being propagated to the log.
My issue was a mistake in the server name, but I think #DomPazz's answer is still useful and the comments provided me with several alternatives that would form a suitable answer to this question:
1. Rename statement
data _null_;
rc = rename("\\servername\Reports\Reporting Templates\Template.xlsm",
"\\servername\Output\Template_new.xlsm",
"file");
put rc=;
run;
2. FCOPY with filename statements
NOTE: When I used FCOPY to move a macro-enabled worksheet (.xlsm) it corrupted the file and I had to recopy it with rename()
filename src "\\servername\Reports\Reporting Templates\Template.xlsm",
filename dest "\\servername\Output\Template_new.xlsm"
data _null_;
rc = fcopy('src', 'dest');
put rc=;
run;
3. Binary Copy
Another method that I haven't test, but copies files over in chunks Rather than byte by byte. One that uses the %mp_binarycopy() macro. Github repo is here:
https://github.com/sasjs/core/blob/main/base/mp_binarycopy.sas
And another similar method described in this SAS blog post, with link to code:
https://blogs.sas.com/content/sasdummy/files/2013/09/binaryfilecopy.sas_.txt