Sorry for the English language skills!
SAS version 9.3
It is necessary to automate the process of obtaining the source code of Job. I know the way through deploy/redeploy, but it is not suitable for the creation of deploy metadata.
Through macro:
data job_source_code;
length uri source_uri $256.;
length text $1000.;
_rc = metadata_getnobj("omsobj:Job?#Name='JOB_NAME'", 1, uri);
_rc = METADATA_GETNASN(uri, 'SourceCode', 1, source_uri);
_rc = METADATA_GETATTR(source_uri, 'StoredText', text);
run;
But the field text is always empty.
What am I doing wrong? Is there any other way to automate the process of obtaining the source code of Job?
I know it's too late that I am answering the question. Recently I have got the same request, before trying did a search in the Web to see if I can get any code. But was not able to find any, luckily I found your code and worked on completing the remaining part. Thanks.
data server_details_in_smc_1;
length uri $256 Name PublicType TransId_uri $100 text f_Direct SourceCode_Location $1000.;
nobj=1;
n=1;
do while(nobj >= 0);
n=n+1;
nobj=metadata_getnobj("omsobj:Job?#Id contains '.'",n,uri);
if (nobj > 0) then
do;
arc=metadata_getattr(uri,"Name",Name);
arc=metadata_getattr(uri,"PublicType",PublicType);
TransId_obj=metadata_getnasn(uri,'SourceCode',1,TransId_uri);
arc = metadata_getnasn(TransId_uri,"Directories",1,f_Direct);
arc = metadata_getattr(f_Direct,"DirectoryName",SourceCode_Location);
output ;
end;
end;
keep Name PublicType SourceCode_Location ;
run;
You can use batch file to deploy jobs from commend line.
I've never done this. But you can find description of this procedure in DI Stuio User's Guide:
http://support.sas.com/documentation/cdl/en/etlug/65807/HTML/default/viewer.htm#p1jxhqhaz10gj2n1pyr0hbzozv2f.htm
Related
Is it possible to get the list of libraries assigned (pre & non-pre assigned) to an application server in SAS Metadata?
I can use dictionary.libnames but it lists only pre-assigned libraries.
Assuming you want to just find out all of the available libraries, and have an account (such as sasadm#saspw) which can see them, then you should be able to iterate using the metadata_getnobj function. Something like this:
nobj=metadata_getnobj("omsobj:SASLibrary?#Id contains '.'",n,uri);
The example from the documentation otherwise should match what you're doing:
data _null_;
length uri $256;
nobj=0;
n=1;
/* Determine how many machine objects are in this repository. */
nobj=metadata_getnobj("omsobj:SASLibrary?#Id contains '.'",n,uri);
put nobj=; /* Number of machine objects found. */
put uri=; /* URI of the first machine object. */
run;
You could then iterate through those, with a do n = 1 by 1 until (n lt 0); loop or similar, and use the metadata_getattr function to obtain the information you want about each uri. You could look at this SAS Communities question for example; the code there should work (their issue was not the code, but their machine setup). Something like this:
data _null_;
length uri $256;
nobj=0;
n=1;
uri=' ';
length name engine libref $256;
call missing(of name engine libref);
nobj=metadata_getnobj("omsobj:SASLibrary?#Id contains '.'",n,uri);
/* Determine how many machine objects are in this repository. */
do n = 1 to nobj;
nobj=metadata_getnobj("omsobj:SASLibrary?#Id contains '.'",n,uri);
rc=metadata_getattr(uri,'Name',name);
rc=metadata_getattr(uri,'Engine',engine);
rc=metadata_getattr(uri,'Libref',libref);
put name= engine= libref=;
end;
run;
This would only include metadata libraries - not libraries that are active, but defined only in SAS code. For the latter, you do need to use dictionary.libnames.
The fastest approach for this, if you have a LOT of libraries, is to use proc metadata.
The below is an extract from a SASjs core macro (this one: https://github.com/sasjs/core/blob/main/meta/mm_getlibs.sas)
/* get list of libraries */
filename response temp;
proc metadata in=
'<GetMetadataObjects>
<Reposid>$METAREPOSITORY</Reposid>
<Type>SASLibrary</Type>
<Objects/>
<NS>SAS</NS>
<Flags>%eval(2048+256+8)</Flags>
<Options/>
</GetMetadataObjects>'
out=response;
run;
/* create an XML map to read the response */
filename sxlemap temp;
data _null_;
file sxlemap;
put '<SXLEMAP version="1.2" name="SASLibrary">';
put '<TABLE name="SASLibrary">';
put '<TABLE-PATH syntax="XPath">//Objects/SASLibrary</TABLE-PATH>';
put '<COLUMN name="LibraryId">><LENGTH>17</LENGTH>';
put '<PATH syntax="XPath">//Objects/SASLibrary/#Id</PATH></COLUMN>';
put '<COLUMN name="LibraryName"><LENGTH>256</LENGTH>>';
put '<PATH syntax="XPath">//Objects/SASLibrary/#Name</PATH></COLUMN>';
put '<COLUMN name="LibraryRef"><LENGTH>8</LENGTH>';
put '<PATH syntax="XPath">//Objects/SASLibrary/#Libref</PATH></COLUMN>';
put '<COLUMN name="Engine">><LENGTH>12</LENGTH>';
put '<PATH syntax="XPath">//Objects/SASLibrary/#Engine</PATH></COLUMN>';
put '</TABLE></SXLEMAP>';
run;
libname _XML_ xml xmlfileref=response xmlmap=sxlemap;
/* sort the response by library name */
proc sort data=_XML_.saslibrary out=work.metalibs;
by libraryname;
run;
To find the ones assigned to a specific app server, as Joe also mentions, you'd need to iterate further with each library id to get the attributes. For that it can help to have a metadata browser. If you don't have Base SAS (which has metabrowse built in) feel free to contact me and I'll send you a tool for that.
Joe's answer can be further condensed as
/* Create metadata libraries listing/inventory */
data META_LIBS (drop=i rc ouri);
length NAME $256 LIBREF $8 ouri $35;
call missing(of _char_);
do i=1 by 1 while(metadata_getnobj("omsobj:SASLibrary?#Id contains '.'", i, ouri) > 0);
rc = metadata_getattr(ouri, 'Name', NAME);
rc = metadata_getattr(ouri, 'Libref', LIBREF);
output;
end;
run;
See its detailed explanation in this section Creating metadata libraries inventory and identifying duplicate LIBREF.
This is a follow-up to my recent question on calculating md5 hash in SAS and python. So, I'm using SAS v9.2 and there is an md5 hash function which takes in a string and returns a hash. What I'd really like though is a way to compute the hash for the file as a whole. Given that I have a hash for each record , is there any way to do this and have the file hash match up with the value obtained by using , say, python code. Taking the sashelp.shoes dataset as an example I exported this to a CSV file and manually removed double quotes and dollars and commas of the currency fields. I then computed the hash for the file as a whole using this python code:
filename = "f:/test/shoes.csv"
md5_hash = hashlib.md5()
with open(filename,"rb") as f:
# Read and update hash string value in blocks of 4K
for byte_block in iter(lambda: f.read(1024*1024),b''):
md5_hash.update(byte_block.replace(b'\r', b'').replace(b'\n', b''))
print(md5_hash.hexdigest())
And got this hash back as output:
f7f205b5b844bf57f5f51685969e0df0
If anyone can replicate this final hash value in SAS for that dataset that would be great.
PS I'm on SAS V9.2
You have two options:
Implement the MD5 algorithm in SAS. I'm aware of existing implementations for SHA and CRC but I'm not sure about MD5.
Call an external utility from SAS to calculate the md5 hash for the file. There is an example here.
My earlier note on limitations applies only when working with DS1. There is no way around the length restriction in DS1. You could try this and you will get an error:
data test;
length x $30000;
x = repeat('-', 30000);
run;
data _null_;
set test;
format m $hex32.;
m = md5(catx(',', x, x));
put m=;
run;`
But Robert Pendridge is correct to point out that DS2 can solve this issue.
%let reclen = 201; /* Length of each record */
%let records = 2000; /* Number of records */
%let totlen = %eval(&reclen * &records);
proc ds2;
data _null_;
retain m;
dcl char(&totlen) m;
method run();
dcl char(200) c;
set shoes;
c = catx(',',&varstr2);
m = strip(m)|| strip(c);
end;
method term();
dcl char(32) hh;
hh = put(md5(m), $hex32.);
put hh=;
end;
enddata;
run;
quit;
This is essentially doing what the Python code is doing. The update merely concatenates the strings and applies the hash. You may have to tighten this up a little bit to remove any extraneous spaces etc., but should work.
Unfortunately you cannot in DS1. The reason is that the maximum variable size that SAS allows is only 32,767 bytes long. You could group the variables in multiple variables, but still when you try to concatenate them (even directly when invoking the md5 function), it will end up truncating it. Your best bet is writing the output to an external text file (as shown below based on your previous example) and generating md5sum on it. This is actually just one little extra step.. You could just use the X command to do that from within SAS itself (provided you are configured to do so).
filename ff "contents.txt" TERMSTR=CR;
data _null_;
set shoes end = lastrec;
newvar2 = catx(',',&varstr2);
file ff;
put newvar2;
run;
I need to read the file name in JCL and find specific string in that. If string present then i need to set the flag variable.
Example:
000063 //SETVARS SET RUNMODE=AY,
000064 // MGRTMODE=M,
000065 // PARMFILE=BASXXXX.T1.XXXX.JIRA.T011746
If "PARMFILE" contains "JIRA", then set JIRAFLAG = Y else JIRAFLAG = N. I need to pass the JIRAFLAG to another JCL/SAS job to process further.
Thanks!
Bharathi
SAS:
000083 /**/
000084 %LET CHKFILE = %SCAN(&SYSPARM,1,+);
000085 %put &CHKFILE ;
000086 %GLOBAL JIRAFLG ;
000087 %MACRO CHK ;
000088 %LET TSTVAL = %SCAN(&CHKFILE,4,.) ;
000089 %PUT &TSTVAL;
000090 %IF &TESTVAL EQ 'JIRA' %THEN %LET JIRAFLG = 'Y' ;
000091 %ELSE %LET JIRAFLG = 'N' ;
000092 %PUT &JIRAFLG ;
000093 %MEND CHK;
000094 %CHK ;
JCL does not provide a scripting language per se although it is intuitive to think of it that way because of the JCL Symbols and conditionals.
I'm not a SAS buy but it seems like your SAS sample does the scanning. I'd suggest that you have a STEP where you parse the PARMFILE statement and set a known return code of your choice. For instance, choose a return code of 1.
Follow that with another step that when the return code is 1 executes the program you want to process when JIRA is in the PARMFILE. Since you mentioned a seperate job you could submit another job through INTRDR.
There are ways to accomplish what you want you just need to be creative. As I said, the conditionals and JCL Symbols lure many people into thinking scripting which is unfortunately not how it works.
SRCHFOR 'YOURSTRING',W (https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.f54u200/sfcmdfgrr.htm)
and then check here for conditional processing
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieab600/iea3b6_Examples_of_IF_THEN_ELSE_ENDIF_statement_constructs.htm
I'm running a process that lists jobs I want to check the modification date on. I list the jobs in a dataset and then pass these to macro variables with a number.
e.g.
Data List_Prep;
Format Folder
Code $100.;
Folder = 'C:\FilePath\Job ABC'; Code = '01 Job Name.sas'; Output;
Folder = 'C:\FilePath\Job X&Y'; Code = '01 Another Job.sas'; Output;
Run;
%Macro List_Check();
Data List;
Set List_Prep;
Job + 1;
Call Symput (Cats("Folder", Job), Strip(Folder));
Call Symput (Cats("Code", Job), Strip(Code));
Run;
%Put Folder1 = &Folder1;
%Put Folder2 = &Folder2;
%MEnd;
%List_Check;
It prints the %Put statement just fine for foler 1, but folder 2 doesn't work right.
Folder1 = C:\FilePath\Job ABC
WARNING: Apparent symbolic reference Y not resolved.
Folder2 = C:\FilePath\Job X&Y
When I then go in to a loop to check the datasets, again, it work, so looks for Folder1, Code1 etc, but I still get the warnings.
How can I stop these warnings? I've tried %Str("&") instead, but still get the issue.
The %superq() macro function is a great way to mask macro triggers that are already in a macro variable. You could either remember to quote the values when using them,
%put Folder1 = %superq(Folder1) ;
or you could adjust your process to quote them right after creating them.
data List_Prep;
length Folder Code $100;
Folder = 'C:\FilePath\Job ABC'; Code = '01 Job Name.sas'; Output;
Folder = 'C:\FilePath\Job X&Y'; Code = '01 Another Job.sas'; Output;
run;
data List;
set List_Prep;
Job + 1;
length dummy $200 ;
call symputx(cats("Folder", Job), Folder);
dummy = resolve(catx(' ','%let',cats("Folder", Job),'=%superq(',cats("Folder", Job),');'));
call symputx(cats("Code", Job), Code);
dummy = resolve(catx(' ','%let',cats("Code", Job),'=%superq(',cats("Code", Job),');'));
drop dummy;
run;
P.S. Don't use FORMAT to define variables. Use statements like LENGTH or ATTRIB that are designed for defining variables. FORMAT is for attaching formats to variable, not for defining them. The only reason that using FORMAT worked is that it had the side effect of SAS defining the variable's type and length to match the format that you attached to it because it was the first place you referenced the variable in the data step.
You can prevent SAS from trying to resolve the ampersand in the value by using the %superq function
%put Folder2 = %superq(Folder2);
I was assigned a task that I don’t know where to start. Here’s the context:
There’s a variable in the data, say VAR1, indicating the directory to a bunch of image files. So for observation 1, VAR1 may look like D:\Project\Data\Images\Image1.tiff and so on. Of course, those image files exist in the computer.
What I need to do is to figure out SAS program(s) and later run them automatically using batch file. When the batch file runs, it will, in some way, opens the image files one by one. By “one by one”, I mean it firsts open one image file and, upon closing that file, it opens the next image file until the end of the list.
Better yet, the batch file will make a copy of the original image files and put them in some folder (e.g., D:\Project\Data\Temp images) before opening them. That is to make sure original data is left untouched.
Do you know how I can write such a program in SAS? I was given the following SPSS file for reference, which does that job nicely as described. I don’t know enough SPSS to understand every detail how it works. The two variables dir5 and tiff5 specify the location of the image files, and variables SCQID and ohhscqid are just ID variables.
string out2 (a200).
compute out2=concat('copy "', ltrim(rtrim(dir5)),"\", tiff5, '"',' "c:\temp\temp.tiff"').
write outfile='E:\Data\Outcome.bat'/'#echo SCQ ID ' ohhscqid .
write outfile='E:\Data\Outcome.bat'/out2.
write outfile='E:\Data\Outcome.bat'/'#"C:\Program Files\Microsoft Office\Office14\OIS.exe" "c:\temp\temp.tiff"'.
execute.
I did the homework and figured out one way that works as I want it to. Not the optimal way programmingly though, but the idea is like this.
data batwide;set have;
echo = '#echo SCQ ID '||ohhscqid;
predir = 'copy '||'"'||strip(dir5)||strip('\')||strip(tiff5)||strip('"');
preexec = '#'||strip('"')||strip('C:\Program Files\Microsoft Office\Office14\OIS.exe')||strip('"');
temp = '"'||strip('c:\temp\temp.tiff')||strip('"');
run;
data batwide; set batwide;
dir = catx(' ',predir,temp);
exec = catx(' ',preexec,temp);
run;
data batlong;set batwide;
format bat $200.;
bat = echo;output;
bat = dir;output;
bat = exec;output;
keep bat;
run;
data _null_;
set batlong;
file "E:\SAS codes and files\batchfile.bat";
put bat;
run;
Sounds like you are asking how to generate a series of OS commands into a text file? You can use a DATA step for that.
If you want to test if the specified files exist then use the FILEEXIST() function.
So if you have SAS dataset name HAVE with a variable named VAR1 that contains the filename then you probably want a program like this:
data _null_;
set have ;
file 'E:\Data\Outcome.bat';
if fileexist(VAR1) then do;
target=catx('\','D:\Project\Data\Temp images',scan(VAR1,-1,'\'));
put 'copy ' VAR1 :$quote. target :$quote. ;
put '"C:\Program Files\Microsoft Office\Office14\OIS.exe" ' target :$quote.;
end;
else putlog 'WARNING: File not found. ' VAR1=;
run;
I don't know SPSS, but will give you an example using unix commands, you can change them to Windows commands and probably do what you described.
In this example I'll only copy some files, but the logic to "open the files one by one" is the same. You will have to play with the code and adjust it to Windows.
First of all, we looking for csv files inside the /home/user directory. Again, adjust the command to windows.
This will create a sas dataset with all the files
filename dirlist pipe "find /home/user/ | grep csv";
data dirlist ;
infile dirlist lrecl=200 truncover;
input line $200.;
file_name = strip(line);
keep file_name;
run;
Then I'll create a macro variable with the file count, I'll all it cntfiles
proc sql noprint;
select count(*) into: cntfiles from dirlist;
quit;
%let cntfiles=&cntfiles;
%put cntfiles=&cntfiles;
The last thing I'm doing is, I'm looping, getting the filenames one by one and copying them to a new macro variable called ©to
This data step (null) will only copy the files, if you want to do something else with them, you'll have to write the code for it.
%macro process_files;
%let copyto = /home/des/33889897/copyto;
%do i=1 %to &cntfiles;
data _null_;
set dirlist (firstobs=&i.);
put file_name=;
call system("cp -f " || file_name || " ©to");
stop;
run;
%end;
%mend process_files;
%process_files;
Take a look at this link, maybe it can help you.
sample code I use frequently to parse list of files in a directory and extract metadata from the file names. Often feeds a step to generate a sequence of macro variables to use in a macro loop to process each file in turn. Just add any substringing of filenames to extract structured content as with the datetxt and date assigment statements in the example where the filename has a datestamp in it that I want to use.
%let extension=txt;
filename infiles "c:\a\b\c";
Data List_of_files
Not_ext
;
Length
path $255
filename $255
extension $10
;
d_open=dopen("infiles") ;
path=pathname("infiles") ;
nfiles=dnum(d_open) ;
do i=1 to nfiles;
filename =dread(d_open,i) ;
extension=scan(filename,-1,'.') ;
datetxt=scan(filename,2,"_");
date=input(scan(filename,2,"_"),date9.);
if upcase(extension) ne "%upcase(&extension)"
then output Not_ext ;
else output list_of_files ;
end;
d_open=dclose(d_open) ;
keep
filename
path
extension
;
Run ;
filename infiles clear;