I am trying to find the optimized way to do this :
I want to delete from a character variable all the observations STARTING with different possible strings such as :
"Subtotal" "Including:"
So if it starts with any of these values (or many others that i didn't write here) then delete them from the dataset.
Best solution would be a macro variable containing all the values but i don't know how to deal with it. (%let list = Subtotal Including: but counts them as variables while they are values)
I did this :
data a ; set b ;
if findw(product,"Subtotal") then delete ;
if findw(product,"Including:") then delete;
...
...
Would appreciate any suggestions !Thanks
First figure out what SAS code you want. Then you can begin to worry about how to use macro logic or macro variables.
Do you just to exclude the strings that start with the values?
data want ;
set have ;
where product not in: ("Subtotal" "Including");
run;
Or do you want to subset based on the first "word" in the string variable?
where scan(product,1) not in ("Subtotal" "Including");
Or perhaps case insensitive?
where lowcase(scan(product,1)) not in ("subtotal" "including");
Now if the list of values is small enough (less than 64K bytes) then you could put the list into a macro variable.
%let list="Subtotal" "Including";
And then later use the macro variable to generate the WHERE statement.
where product not in: (&list);
You could even generate the macro variable from a dataset of prefix values.
proc sql noprint;
select quote(trim(prefix)) into :list separated by ' '
from prefixes
;
quit;
Related
I have a project with multiple programs. Each program has a proc SQL statement which will use the same list of values for a condition in the WHERE clause; however, the column type of one database table needed is a character type while the column type of the other is numeric.
So I have a list of "Client ID" values I'd like to put into a macro variable as these IDs can change, and I would like to change them once in the variable instead of in multiple programs.
For example, I have this macro variable set up like so and it works in the proc SQL which queries the character column:
%let CLNT_ID_STR = ('179966', '200829', '201104', '211828', '264138');
Proc SQL part:
...IN &CLNT_ID_STR.
I would like to create another macro variable, say CLNT_ID_NUM, which takes the first variable (CLNT_ID_STR) but removes the quotes.
Desired output: (179966, 200829, 201104, 211828, 264138)
Proc SQL part: ...IN &CLNT_ID_NUM.
I've tried using the sysfunc, dequote and translate functions but have not figured it out.
TRANSLATE doesn't seem to want to allow a null string as the replacement.
Below uses TRANSTRN, which has no problem translating single quote into null:
1 %let CLNT_ID_STR = ('179966', '200829', '201104', '211828', '264138');
2 %let want=%sysfunc(transtrn(&clnt_id_str,%str(%'),%str())) ;
3 %put &want ;
(179966, 200829, 201104, 211828, 264138)
It uses the macro quoting function %str() to mask the meaning of a single quote.
Three other ways to remove single quotes are COMPRESS, TRANSLATE and PRXCHANGE
%let CLNT_ID_STR = ('179966', '200829', '201104', '211828', '264138');
%let id_list_1 = %sysfunc(compress (&CLNT_ID_STR, %str(%')));
%let id_list_2 = %sysfunc(translate(&CLNT_ID_STR, %str( ), %str(%')));
%let id_list_3 = %sysfunc(prxchange(%str(s/%'//), -1, &CLNT_ID_STR));
%put &=id_list_1;
%put &=id_list_2;
%put &=id_list_3;
----- LOG -----
ID_LIST_1=(179966, 200829, 201104, 211828, 264138)
ID_LIST_2=( 179966 , 200829 , 201104 , 211828 , 264138 )
ID_LIST_3=(179966, 200829, 201104, 211828, 264138)
It really doesn't matter that TRANSLATE replaces the ' with a single blank () because the context for interpretation is numeric.
I have the following code where I rename column names; I would like to keep only the variables created by the macro. I do realize I can drop the old variables but am curious if there is a keep option I can place inside the macro.
So for example, in the datastep, I would want to keep only the variable that start with '%transform_this(JUNE19)';
Thanks!
%macro transform_this(x);
&x._Actual=input(Current_Month, 9.0);
&x._Actual_Per_Unit = input(B, 9.);
&x._Budget=input(C, 9.);
&x._Budget_Per_Unit=input(D, 9.);
&x._Variance=input(E, 9.);
&x._Prior_Year_Act=input(G, 9.);
Account_Number=input(H, 9.);
Account_Description=put(I, 35.);
&x._YTD_Actual=input(Year_to_Date, 9.);
&x._YTD_Actual_Per_Unit=input(L, 9.);
%mend transform_this;
data June_53410_v1;
set June_53410;
%transform_this(JUNE19);
if Account_Description='Account Description' then DELETE;
Drop Current_Month B C D E G H I Year_to_Date L M N;
run;
keep June19_: Account_:;
This keeps all variables starting with June19_ and Account_ which are the ones you need evidently.
am curious if there is a keep option I can place inside the macro.
You can definitely use keep in your macro:
%macro transform_this(x);
keep &x._Actual &x._Actual_Per_Unit
&x._Budget &x._Budget_Per_Unit
&x._Variance &x._Prior_Year_Act
Account_Number Account_Description
&x._YTD_Actual &x._YTD_Actual_Per_Unit
;
&x._Actual=input(Current_Month, 9.0);
/* ...and the rest of your code */
%mend transform_this;
Any reason you thought you can't?
Add two sentinel variables to the data step, one before the macro call and one after. Use the double dash -- variable name list construct in a keep statement and drop the sentinels in the data step output data set specifier drop= option.
data want (drop=sentinel1 sentinel2); /* remove sentinels */
set have;
retain sentinel1 0;
%myMacro (…)
retain sentinel2 0;
…
keep sentinel1--sentinel2; * keep all variables created by code between sentinel declarations;
run;
Name Range Lists
Name range lists rely on the order of variable definition, as shown in
the following table:
Name Range Lists
Variable List Included Variables
x -- a all variables in order of variable definition, from
variable x to variable a inclusive
x -NUMERIC- a all numeric variables from variable x to variable a inclusive
x -CHARACTER- a all character variables from variable x to variable a inclusive
Note: Notice that name range lists use a double hyphen ( -- ) to designate
the range between variables, and numbered range lists use a single
hyphen to designate the range.
First i have created this table
data rmlib.tableXML;
input XMLCol1 $ 1-10 XMLCol2 $ 11-20 XMLCol3 $ 21-30 XMLCol4 $ 31-40 XMLCol5 $ 41-50 XMLCol6 $ 51-60;
datalines;
| AAAAA A||AABAAAAA|| BAAAAA|| AAAAAA||AAAAAAA ||AAAA |
;
run;
I want to clean, concatenate and export. I have written the following code
data rmlib.tableXML_LARGO;
file CleanXML lrecl=90000;
set rmlib.tableXML;
array XMLCol{6} ;
array bits{6};
array sqlvars{6};
do i = 1 to 6;
*bits{i}=%largo(XMLCol{i})-2;
%let bits =input(%largo(XMLCol{i})-2,comma16.5);
sqlvars{i} = substr(XMLCol{i},2,&bits.);
put sqlvars{i} &char10.. #;
end;
run;
the macro largo count how many characters i have
%macro largo(num);
length(put(&num.,32500.))
%mend;
What i need is instead of have char10, i would like that this number(10) would be the length, of each string, so to have something like
put sqlvars{i} &char&bits.. #;
I don't know if it possible but i can't do it.
I would like to see something like
AAAAA AAABAAAAA BAAAAA AAAAAAAAAAAAA AAAA
It is important to me to keep the spaces(this is only an example of an extract of a xml extract). In addition I will change (for example) "B" for "XPM", so the size will change after cleaning the text, that it what i need to be flexible in the char
Thank you for your time
Julen
I'm still not quite sure what you want to achieve, but if you want to combine the text from multiple varriables into one variable, then you could do something along the lines:
proc sql;
select name into :names separated by '||'
from dictionary.columns
where 1=1
and upcase(libname)='YOURLIBNAME'
and upcase(memname)='YOURTABLENAME';
quit;
data work.testing;
length resultvar $ 32000;
set YOURLIBNAME.YOURTABLENAME;
resultvar = &names;
resultvar2 = compress(resultvar,'|');
run;
Wasn't able to test this, but this should work if you replace YOURLIBNAME and YOURTABLENAME with your respective tables. I'm not 100% sure if the compress will preserve the spaces in the text.. But I think it should.
The format $VARYING. <length-variable> is a good candidate for solving this output problem.
On the presumption of having a number of variables whose values are vertical-bar bounded and wanting to output to a file the concatenation of the values without the bounding bars.
data have;
file "c:\temp\want.txt" lrecl=9000;
length xmlcol1-xmlcol6 $100;
array values xmlcol1-xmlcol6 ;
xmlcol1 = '| A |';
xmlcol2 = '|A BB|';
xmlcol3 = '|A BB|';
xmlcol4 = '|A BBXC|';
xmlcol5 = '|DD |';
xmlcol6 = '| ZZZ |';
do index = 1 to dim(values);
value = substr(values[index], 2); * ignore presumed opening vertical bar;
value_length = length(value)-1; * length with still presumed closing vertical bar excluded;
put value $varying. value_length #; * send to file the value excluding the presumed closing vertical bar;
end;
run;
You have some coding errors in that is making it difficult to understand what you want to do.
Your %largo() macro doesn't make any sense. There is no format 32500.. The only reason it would run in your code is because you are trying to apply the format to a character variable instead of a number. So SAS will automatically convert to use the $32500. instead.
The %LET statement that you have hidden in the middle of your data step will execute BEFORE the data step runs. So it would be less confusing to move it before the data step.
So replacing the call to %largo() your macro variable BITS will contain this text.
%let bits =input(length(put(XMLCol{i},32500.))-2,comma16.5);
Which you then use inside a line of code. So that line will end up being this SAS code.
sqlvars{i} = substr(XMLCol{i},2,input(length(put(XMLCol{i},$32500.))-2,comma16.5));
Which seems to me to be a really roundabout way to do this:
sqlvars{i} = substr(XMLCol{i},2,length(XMLCol{i})-2);
Since SAS stores character variables as fixed length, it will pad the value stored. So what you need to do is to remember the length so that you can use it later when you write out the value. So perhaps you should just create another array of numeric variables where you can store the lengths.
sqllen{i} = length(XMLCol{i})-2;
sqlvars{i} = substr(XMLCol{i},2,sqllen{i});
I have a table like this :
|Num | Label
-----------------------
1|1 | a thing
2|2 | another thing
3|3 | something else
4|4 | whatever
I want to replace my values of my label column by something more generic for example the first two lines : label One, the two next ones label Two ...
|Num | Label
-----------------------
1|1 | label One
2|2 | label One
3|3 | label Two
4|4 | label Two
How can I do that using proc format procedure ? I was wondering if I can use either the number of lines or another column like Num.
I need to do something like this :
proc format;
value label_f
low-2 = "label One"
3-high = "label Two"
;
run;
But I want to specify the number of the line or the value of the Num column.
You could do what you are describing using the words format. You could swap out num for _N_ in the ceil function below in order to use the observation number instead of the value of num (if they are not always equal):
data have;
length num 8 label $20;
infile datalines dlm='|';
input num label $;
datalines;
1|a thing
2|another thing
3|something else
4|whatever
5|whatever else
6|so many things
;
run;
data want;
set have;
label=catx(' ','label',propcase(put(ceil(num/2),words.)));
run;
Although this answer is probably a bit too specific to your example and it may not apply in your actual context.
Gatsby:
It sounds like you want to format NUM instead of LABEL.
Where you want the use the 'generic' representation defined by your format simply place a FORMAT statement in the Proc being used:
PROC PRINT data=have;
format num label_f.;
RUN;
If you want both num and generic, you will need to add a new column to the data for use during processing. This can be done with a view:
data have_view / view=have_view;
set have;
num_replicate1 = num;
attrib num_replicate1 format=label_f. label='Generic';
num_replacement = put (num,label_f.);
attrib num_replacement label='Generic'; %* no format because the value is the formatted value of the original num;
run;
PROC PRINT data=have_view;
var num num_replicate1 num_replacement;
RUN;
If you want a the 'generic' representation of the NUM column to be used in by-processing as a grouping variable, you have several scenarios:
know apriori the generic representation is by-group clustered
use a view and process with BY or BY ... NOTSORTED if clusters are not in sort order
force ordering for use with by-group processing
use an ordered SQL view containing the replicate and process with BY
add a replicate variable to the data set, sort by the formatted value and process with BY
A direct backmap from label to num to generic is possible only if the label is known to be unique, or you know apriori the transformation backmap-num + num-map is unique.
Proc FORMAT also has a special value construct [format] that can be used to map different ranges of values according to different formatting rules. The other range can also map to a different format that itself has an other range that maps to yet another different format. The SAS format engine will log an error if you happen to define a recursive loop using this advanced kind of format mapping.
propaedeutics
One of my favorite Dorfman words.
Format does not replace underlying values. Format is a map from the underlying data value to a rendered representation. The map can be 1:1, many:1. The MultiLabel Format (MLF) feature of the format system can even perform 1:many and many:many mappings in procedures many MLF enabled procedures (which is most of them)
To replace an underlying value with it's formatted version you need to use the PUT, PUTC or PUTN functions. The PUT functions always outputs a character value.
character ⇒ PUT ⇒ character [ FILE / PUT ]
numeric ⇒ PUT ⇒ character [ FILE / PUT ]
There is no guarantee a mapped value will mapped to the same value, it depends on the format.
INFORMATs are similar to FORMATs, however the target value depend on the in format type
character ⇒ INPUT ⇒ character [ INFILE / INPUT ]
numeric ⇒ INPUT ⇒ character
character ⇒ INPUT ⇒ numeric [ INFILE / INPUT ]
numeric ⇒ INPUT ⇒ numeric
Custom formats are created with Proc FORMAT. The construction of a format is specified by either the VALUE statement, or the CNTLIN= option. CNTLIN lets you create formats directly from data and avoids really large VALUE statements that are hand-entered or code-generated (via say macro)
Data-centric 'formatting' performs the mapping through a left-join. This is prevalent in SQL data bases. Left-joins in SAS can be done through SQL, DATA Step MERGE BY and FORMAT application. 1:1 left-joins can also be done via Hash object SET POINT=
In a basic data step I'm creating a new variable and I need to filter the dataset based on this new variable.
data want;
set have;
newVariable = 'aaa';
*lots of computations that change newVariable ;
*if xxx then newVariable = 'bbb';
*if yyy AND not zzz then newVariable = 'ccc';
*etc.;
where newVariable ne 'aaa';
run;
ERROR: Variable newVariable is not on file WORK.have.
I usually do this in 2 steps, but I'm wondering if there is a better way.
( Of course you could always write a complex where statement based on variables present in WORK.have. But in this case the computation of newVariable it's too complex and it is more efficient to do the filter in a 2nd data step )
I couldn't find any info on this, I apologize for the dumb question if the answer is in the documentation and I didn't find it. I'll remove the question if needed.
Thanks!
Use a subsetting if statement:
if newVariable ne 'aaa';
In general, if <condition>; is equivalent to if not(<condition>) then delete;. The delete statement tells SAS to abandon this iteration of the data step and go back to the start for the next iteration. Unless you have used an explicit output statement before your subsetting if statement, this will prevent a row from being output.