How can I remove the space at the end of the output? I am getting a space at the end within quotes
data _null_;
files = 'AAAAAAAAAAAAAA,BBBBBBBBBBBBBB';
f_count = countw(files);
do i=1 to f_count;
file = scan(files, i, ',');
put '''file''';
end;
run;
output: There is a space at the end
'AAAAAAAAAAAA '
'BBBBBBBBBBBB '
That is because you are used LIST MODE style PUT statement. SAS will write a delimiter (space in this case) after each variable written when using LIST MODE style in PUT statements.
You could just use a cursor movement command to back-up one byte so that the closing quote is written over the space.
put "'" file +(-1) "'";
You could add the quotes to the variable rather than in the PUT statement. (Then the space will be written after the closing quote.)
file = quote(strip(scan(files, i, ',')),"'");
put file ;
Or you could use the $VARYING format to write the exact number of bytes that FILE contains.
len = lengthn(file);
put "'" file $varying200. len "'" ;
If you don't mind using double quote characters instead of single quotes you could just use the $QUOTE format.
put file :$quote. ;
You could also use the DSD option on the FILE statement. SAS will then automatically add double quotes if they are needed. They will be needed when the value contains the delimiter character or quote characters themselves. With the DSD option in effect you can use the ~ modifier in the PUT statement to write quotes around the value even when the value does not require quoting.
data _null_;
file log dsd ;
files = 'AAAAAAAAAAAAAA,BBBBBBBBBBBBBB';
f_count = countw(files);
do i=1 to f_count;
file = scan(files, i, ',');
put file ~;
end;
run;
Related
I want to replace certain values in my json file (in this example null values with empty quotation marks.) My solution is working correctly but, for some mysterious reason, the last character of the json file is deleted. Regardless of the last character, the code always deletes it - I have also tried with a different json file that ends in curly braces.
What is causing this and more importantly how can I prevent this?
data testdata_;
input var1 var2 var3;
format _all_ commax10.1;
datalines;
3.1582 0.3 1.8
21 . .
1.2 4.5 6.4
;
proc json out = 'G:\test.json' pretty fmtnumeric nosastags keys;
export testdata_;
run;
data _null_;
infile 'G:\test.json';
file 'G:\test.json';
input;
_infile_ = tranwrd(_infile_,'null','""');
put _infile_ ;
run;
To see how the contents change, first run the code until "data null" statement and check the file content, then run the last statement.
Data _null_ has it correct; don't write to the same file. SAS offers this option, but in the modern day it's almost always the wrong answer, due to how SAS supports this and the fact that storage is sufficiently cheap and fast.
In this case, it looks like it's a relatively easy fix, but you probably should do as suggested and write to a new file anyway - there will be other issues.
data testdata_;
input var1 var2 var3;
format _all_ commax10.1;
datalines;
3.1582 0.3 1.8
21 . .
1.2 4.5 6.4
;
proc json out = 'H:\temp\test.json' pretty fmtnumeric nosastags keys;
export testdata_;
run;
data _null_;
infile 'H:\temp\test.json' end=eof;
file 'H:\temp\test.json';
input #;
putlog _infile_;
_infile_ = tranwrd(_infile_,'null','"" ');
len = length(_infile_);
put _infile_ ;
if eof then put _infile_;
run;
There's two changes. One, I use '"" ' instead of '""' in the tranwrd; that's because otherwise you end up with slightly odd results with new lines being added. If your JSON parser doesn't like "" ,, then you may want to instead have two tranwrd, one for null, and one for null, or something similar (or use a regular expression). But what's important is the number of characters needs to match in the input and the output. If you can't handle that (like the extra spaces are problematic) then you're left with "write a new file".
Two, I look for the end of the file, then intentionally write out a second line there. That avoids the issue you're having with the bracket, as it avoids having the EOF being written out before the bracket. I'm not 100% sure I know why you need that - but you do.
Another option, which might make more sense, is to only write the lines that have the bracket.
data _null_;
infile 'H:\temp\test.json' sharebuffers;
file 'H:\temp\test.json';
input #;
putlog _infile_;
if find(_infile_,'null') then do;
_infile_ = tranwrd(_infile_,'null','"" ');
put _infile_;
end;
run;
I added sharebuffers because that should make it run a bit faster. Note that I also remove one space - something weird about how SAS does this seems to otherwise remove a space from the following line otherwise. No idea why, probably something weird with EOL characters.
But again - don't do any of this unless there's no other option. Write a new file.
One strange thing is that the PROC JSON always writes a text file that uses LF as the end of line characters.
So you might be able to get your overwriting of the file to work if add these caveats:
Use TERMSTR=LF on the INFILE statement.
Use SHAREDBUFFERS on the INFILE statement.
Replace the string with the same number of bytes with the TRANWRD() function and not put a space as the last character on the line.
I would also search for ': null' instead of just 'null' to reduce risk of replacing those characters in some other string in the file.
data _null_;
infile json SHAREBUFFERS termstr=lf ;
file json ;
input ;
_infile_ = tranwrd(_infile_,': null',': ""');
put _infile_;
run;
I need to parse a log file to pick out strings that match the following case-insensitive pattern:
libname.data <--- Okay
libname.* <--- Not okay
For those with SAS experience, I'm trying to get SAS dataset names out of a large log.
All strings are space-separated. Some examples of lines:
NOTE: The data set LIBNAME.DATA has 428 observations and 15 variables.
MPRINT(MYMACRO): data libname.data;
MPRINT(MYMACRO): create table libname.data(rename=(var1 = var2)) as select distinct var1, var2 as
MPRINT(MYMACRO): format=date. from libname.data where ^missing(var1) and ^missing(var2) and
What I've tried
This PERL regular expression:
/^(?!.*[.*]{2})[a-z0-9*_:-]+(?:\.[a-z0-9;_:-]+)+$/mi
https://regex101.com/r/jYkXn5/1
In SAS code:
data test;
line = 'words and stuff libname.data';
test = prxmatch('/^(?!.*[.*]{2})[a-z0-9*_:-]+(?:\.[a-z0-9;_:-]+)+$/mi', line);
run;
Problem
This will work when the line only contains this exact string, but it will not work if the line contains other strings.
Solution
Thanks, Blindy!
The regex that worked for me to parse SAS datasets from a log is:
/(?!.*[.*]{3})[a-z_]+[a-z0-9_]+(?:\.[a-z0-9_]+)/mi
data test;
line = 'NOTE: COMPRESSING DATA SET LIBNAME.DATA DECREASED SIZE BY 46.44 PERCENT';
prxID = prxparse('/(?!.*[.*]{3})[a-z]+[a-z0-9_]+(?:\.[a-z0-9_]+)/mi');
call prxsubstr(prxID, line, position, length);
dataset = substr(line, position, length);
run;
This will still pick up some SQL select statements but that is easily solvable through post-processing.
You anchored your expression at the beginning, simply remove the first ^ and you're set.
/(?!.*[.*]{2})[a-z0-9*_:-]+(?:\.[a-z0-9;_:-]+)+$/mi
You can get by just locating the following landmark text in a log file line.
... data set <LIBNAME>.<MEMNAME> ...
If the data set name is in the log you can presume it was correctly formed.
data want;
length line $1000;
infile LOG_FILE lrecl=1000 length=L;
input line $VARYING. L;
* literally "data set <name>" followed by space or period;
rx = prxparse('/data set (.*?)\.(.*?)[. ]/');
if prxmatch(rx,line) then do;
length libname $8 memname $32;
libname = prxposn(rx,1,line);
memname = prxposn(rx,2,line);
line_number = _n_;
output;
end;
keep libname memname line_number;
run;
Some adjustment would be needed if the data set names are name literals of the form '<anything>'N
There are also a plethora of existing SAS Log file parsers and analyzers out on the web that you can utilize.
The lookahead at the start prevents matching .. but the pattern by itself will not match that, as the character classes are repeated 1 or more times and do not contain a dot.
If you don't want to match ** as well, and the string should not start with *, you can add that to a character class [*.] together with the dot, and take it out of the first character class.
In that case, you could omit the positive lookahead and the anchor:
/[a-z0-9_:-]+(?:[.*][a-z0-9_:-]+)+/i
Regex demo
As the pattern does not contain any anchors, you could omit the m flag.
I am trying to generate a script from within SAS. Unfortunately, I need to write the line
strMSHTA = "mshta.exe ""about:<input type=file id=FILE>" _
& "<script>FILE.click();new ActiveXObject('Scripting.FileSystemObject')" _
& ".GetStandardStream(1).WriteLine(FILE.value);close();resizeTo(0,0);</script>"""
To write the script to file, I am using the following approach:
/* write the script to file */
filename outfile ".\temp.vbs";
data _null_;
file outfile;
put 'WScript.Echo "Hello, World!"';
run;
/* run the script */
data _null_;
call system(".\temp.vbs");
run;
/* housekeeping */
%let rc=%sysfunc(fdelete(outfile));
%symdel rc;
filename outfile clear;
Unfortunately, the put statement doesn't take kindly to &, ', and ". I have tried all of the macro quoting functions, but cannot get anything to work. If necessary, I could use the overloaded concatenation operator + instead of &. Doing this leaves only an error with the middle line due to the single quotes on ActiveXObject('Scripting.FileSystemObject'). Any ideas?
Not sure I understand the problem. Using single quotes instead of double quotes on the outside of your literal strings will eliminate the need to worry about macro triggers like & or %, but you will still need to double up any literal single quote characters you need to generate.
put
'strMSHTA = "mshta.exe ""about:<input type=file id=FILE>" _'
/' & "<script>FILE.click();new ActiveXObject(''Scripting.FileSystemObject'')" _'
/' & ".GetStandardStream(1).WriteLine(FILE.value);close();resizeTo(0,0);</script>"""'
;
You could also break your strings literals into multiple string literals and use different outer quotes for each.
put
'strMSHTA = "mshta.exe ""about:<input type=file id=FILE>" _'
/' & "<script>FILE.click();new ActiveXObject('
"'Scripting.FileSystemObject'"
')" _'
/' & ".GetStandardStream(1).WriteLine(FILE.value);close();resizeTo(0,0);</script>"""'
;
I find it good practice to restrict my code to within 80 characters per line. Since SAS ignores white space, this usually isn't a problem. However, I occasionally need to refer to some string which is excessively long.
For example,
filename infile "B:\This\file\path\is\really\long\but\there\is\nothing\I\can\do\about\it\because\it\is\on\a\shared\network\drive\and\I\am\stuck\with\whatever\organization\or\lack\thereof\exists\for\directory\hierarchies\filename.txt";
I can think of two solutions:
1) Insert a carriage return. This however makes the code look quite messy and may unwittingly introduce invisible characters (i.e \r\n) into the string.
filename infile "B:\This\file\path\is\really\long\but\there\is\nothing\
I\can\do\about\it\because\it\is\on\a\shared\network\drive\and\I\am\stuck\
with\whatever\organization\or\lack\thereof\exists\for\directory\hierarchies\
filename.txt";
2) Use macro variables to break the string into several parts.
%let part1 = B:\This\file\path\is\really\long\but\there\is\nothing\;
%let part2 = I\can\do\about\it\because\it\is\on\a\shared\network\drive\and\I\am\stuck\;
%let part3 = with\whatever\organization\or\lack\thereof\exists\for\directory\hierarchies\;
%let part4 = filename.txt;
filename infile "&part1.&part2.&part3.&part4.";
%let path = %sysfunc(pathname(infile));
%put &path;
Ideally, I would like something which allows me to follow the indentation scheme of the rest of the code.
filename infile "B:\This\file\path\is\really\long\but\there\is\nothing\
I\can\do\about\it\because\it\is\on\a\shared\network\drive\and\I\am\stuck\
with\whatever\organization\or\lack\thereof\exists\for\directory\hierarchies\
filename.txt";
A possible solution, at least within the context of this example, would be to bypass a declaration altogether and prompt the use for the input file. This does not appear easy to implement, however.
For this type of situation where the string needs to be used as one token then splitting it into separate macro variables is the best approach.
%let basedir=b:\Main Folder;
%let project=This project\has\many\parts;
%let fname=filename.txt ;
...
infile "&basedir/&project/&fname" ;
Note that SAS is happy to convert your directory delimiters between Unix (/) and Windows (\) style automatically for you.
You could also take advantage of using a fileref to point to a starting point in your directory tree.
filename basedir "&basedir";
...
infile basedir("&project/&fname");
You could also store the path in a text file or dataset and use that to generate the path into a macro variable.
data _null_;
infile 'parameter_file.txt' ;
input filename :$256. ;
call symputx('filename',filename);
run;
...
infile "&filename" ;
Another variation on using macro variable is to use multiple %LET statements to initialize a single macro variable. That way you can break the long string into multiple tokens.
%let fname=B:\This\file\path\is\really\long\but\there\is\nothing;
%let fname=&fname\I\can\do\about\it\because\it\is\on\a\shared\network\drive\and\I\am\stuck;
%let fname=&fname\with\whatever\organization\or\lack\thereof\exists\for\directory\hierarchies;
%let fname=&fname\filename.txt;
Or you could use a DATA step to set your macro variable instead.
data _null_;
call symputx('fname',catx('\'
,'B:\This\file\path\is\really\long\but\there\is\nothing\I\can'
,'do\about\it\because\it\is\on\a\shared\network\drive\and\I\am\stuck'
,'with\whatever\organization\or\lack\thereof\exists\for\directory'
,'hierarchies\filename.txt'
));
run;
For a situation where you need to put a long string in code such as a dataset label or some type of description consider using %cmpres. The function has limits but is useful to keep one inside 80 columns if they can use it. Here, my CR and other adjacent white spaces are being "compressed" in to a single space character.
%macro get_filename(FILEPATH_FILE, FILE)
/DES=%cmpres("returns a file's name, placed into var FILE, removing the
file path from FILEPATH_FILE.");
If you do this a lot, use %SYSFUNC() and COMPRESS() to make a user-defined macro like this:
%macro c(text);
%sysfunc(compress(&text, ,s))
%mend;
filename infile %c("B:\This\file\path\is\really\long\but\there\is\nothing\I\
can\do\about\it\because\it\is\on\a\shared\network\drive\
and\I\am\stuck\with\whatever\organization\or\lack\thereof\
exists\for\directory\hierarchies\and\he\uses\B\as\a\drive\
OMG\who\does\that\filename.txt");
%put %c("B:\This\file\path\is\really\long\but\there\is\nothing\I\
can\do\about\it\because\it\is\on\a\shared\network\drive\
and\I\am\stuck\with\whatever\organization\or\lack\thereof\
exists\for\directory\hierarchies\and\he\uses\B\as\a\drive\
OMG\who\does\that\filename.txt");
Option "s" in the COMPRESS() function removes all whitespace characters.
SAS posts notes on the log, you can ignore them:
NOTE: The quoted string currently being processed has become more than 262 characters long. You might have unbalanced quotation marks.
I have a quick question.
I am learning SAS and have come across the dsd= option.
Does anyone know what this stands for? It might assist in remembering / contextualizing.
Thanks.
Rather than just copy and pasting text from the internet. I'll try to explain it a bit clearer. Like the delimiter DLM=, DSD is an option that you can use in the infile statement.
Suppose a delimiter has been specified with DLM= and we used DSD. If SAS sees two delimiters that are side by side or with only blank space(s) between them, then it would recognize this as a missing value.
For example, if text file dog.txt contains the row:
171,255,,dog
Then,
data test;
infile 'C:\sasdata\dog.txt' DLM=',' DSD;
input A B C D $;
run;
will output:
A B C D
171 255 . dog
Therefore, variable C will be missing denoted by the .. If we had not used DSD, it would return as invalid data.
DSD stands for Delimiter-Sensitive Data.
The DSD (Delimiter-Sensitive Data) in infile statement does three things for you. 1: it ignores delimiters in data values enclosed in quotation marks; 2: it ignores quotation marks as part of your data; 3: it treats two consecutive delimiters in a row as missing value.
Source: easy sas
DSD (delimiter-sensitive data)
specifies that when data values are enclosed in quotation marks,
delimiters within the value are treated as character data. The DSD
option changes how SAS treats delimiters when you use LIST input and
sets the default delimiter to a comma. When you specify DSD, SAS
treats two consecutive delimiters as a missing value and removes
quotation marks from character values.
http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a000146932.htm
DSD refers to delimited data files that have delimiters back to back when there is missing data. In the past, programs that created delimited files always put a blank for missing data. Today, however, pc software does not put in blanks, which means that the delimiters are not separated. The DSD option of the INFILE statement tells SAS to watch out for this. Below are examples (using comma delimited values) to illustrated:
Old Way: 5,4, ,2, ,1 ===> INFILE 'file' DLM=',' ... etc
New Way: 5,4,,2,,1 ===> INFILE 'file' DLM=',' DSD ... etc.
Refer
reference