Consider the following data set test:
x y
1 A
2 A
. A
4 A
. B
. B
7 B
8 B
Basically I want missing values in the group to be replaced by the previous value. But if the previous value is from another group then don't use that value to replace the current value.
Consider this code:
proc sort data=testout=Sorted;
by y;
run;
data Out2(drop=_x);
set Sorted;
by y;
retain _x;
if first.y then do;
_x=x;
end; else do;
if missing(x) then do;
x = _x;
end; else do;
_x = x;
end;
end;
run;
What does the underscore behind the x mean (_x)?
Using an underscore as a prefix is a fairly common naming convention for temporary variables. As far as SAS is concerned, _x is just another variable - the _ doesn't have any special effect. However, if all your temporary variables (and only those ones) start with an underscore, it's a bit less work to tidy up at the end of your data step, because you can use a : wildcard to drop them all in one go. e.g.
data example;
set sashelp.class;
_age = age;
_sex = sex;
drop _: ;
run;
_x is simply a new variable. It could as easily be z or last_x. However, it may be more clear to someone reading the code that it is related to x; while I'm not aware of any convention similar to "_ means previous value of ", it's not an unreasonable one.
Related
if column 'all' contains the string which is in column 'sch', then this string will be replaced by the string in 'rep'. the column 'new_all' is what I expected.
data a0;
input sch $9. rep $14. ;
cards;
map_clm map_claim
xyz_ttt xyz
drug_clm drug_clm_test
fee_sch fee_sch_test
;
run;
data a1;
input all $26. new_all $30.;
cards;
from abc.xyz_ttt d2 left from abc.xyz d2 left
D1.Xwlk,abc.xyz_TTT left D1.xwlk,abc.xyz left
d1.x,abc.map_clms,d2.dos d1.x,abc.map_clms,d2.dos
ABC.XYZ_Ttt left join d1 ABC.xyz left join d1
,tt.drug_CLM, tt.Xyz_ttt ,tt.drug_clm_test, tt.xyz
d3.DOS,t2.fee_SCH,tt.day fd3.DOS,t2.fee_sch_test,tt.day
;
run;
I'm assuming that you want to transform column all into column new_all, using the values from the a0 data set to describe/control the transformation.
%macro do_it;
* Set up some macro variable arrays to hold the from/to pairs;
data _null_;
set a0 end=end;
call symput("sch" || _n_, sch);
call symput("rep" || _n_, rep);
if end then call symput("max", _n_);
run;
* Process the data;
data want;
length new_all_new $ 200;
set a1;
* Copy the incoming value to the output;
new_all_new = all;
* Apply each of the transformations in turn;
%do i = 1 %to &max;
new_all_new = tranwrd(new_all_new, "&&sch&i", "&&rep&i");
%end;
* Did we get the expected value?;
if new_all_new ne new_all then do;
put "ERROR: Did not get expected value: " all= new_all= new_all_new=;
end;
run;
%mend;
%do_it;
The above code should be pretty close, though I'm unable to test it at the moment. This code is case-sensitive; your expected data suggests that you want to apply the transformations case-insensitively, whilst also preserving the case of the rest of the string. That's a little trickier to implement; if it's really a requirement, a regex search and replace (prxparse("s/&&sch&i/&&rep&i/");) might be the best approach.
There's also the issue that you want to replace 'map_clm' but not 'map_clms', which also suggests regexes might be the cleanest solution.
In any case, this code gives a reasonable starting point for you to work from, I think. It's great that you have built-in expected values to test against.
Are you really modifying SQL code programatically?
case when t1.all CONTAINS t1.sch
then t1.rep
end
If you skip 'else' statement, you'll get missing values (null) for new column if the condition is not satisfied. if you want to keep t1.all values, you have to add else t1.all statement before end.
I have two groups, A and B, and two numeric variables, X and Y. I want to create two new variables, new1 and new2, based on the values of X and Y (respectively) for group B (i.e., IF group = B THEN new1 = X, new2 = Y). I want to take those newly created variables, append them to group A, and then delete group B. In the end, there should be one row for group A containing X, Y, new1, and new2. I'm uncertain how to accomplish this.
I've looked into using PROC TRANSPOSE, but I'm unsure if that's the right starting point. My internet searches are lacking because I'm not even sure what to call what I'm attempting to do, though I'm betting this is a common procedure requiring a common solution.
EXAMPLE
Not sure how to generalize the problem, but for the given problem this will work:
/* Just reversing the records */
proc sort data = have;
by descending group;
run;
data want;
set have;
retain new1 new2;
if _N_ = 1 then do;
new1 = x;
new2 = y;
end;
else output;
run;
This sounds like a case of 1 to 1 merging (merge with out BY).
data have; input
group $1. x y; datalines;
A 3 4
B 2 6
run;
data want;
merge
have(where=( group='A'))
have(where=(Bgroup='B') rename=(x=Bx y=By group=Bgroup))
;
drop Bgroup;
run;
My data is more than 70,000. I have more than 50 variables. (Var1 to Var50). In each variable, there are about about 30 groups (I'll use a to z). I am trying to get a selection of data using if statements. I'd like to select every data with the same group. Eg data in var 1 to 30 with a, data with var 1 to 30 in b.
I seem to be writing
If (Var1="a" and Var2="a" and Var3="a" and Var4="a" and all the way to var50=
"a") or (Var1="b" and Var2="a" and Var3="b" and Var4="b" and all the way to var50=
"b")...
How do I consolidate? I tried using an array but it didnt work and i was not sure if arrays work in the IF and then statement.
IF (VAR2="A" or VAR2="B" or VAR2="C" or VAR2="D"
or VAR3="A" or VAR3="B" or VAR3="C" or VAR3="D"
or VAR4="A" or VAR4="B" or VAR4="C" or VAR4="D"
or VAR5="A" or VAR5="B" or VAR5="C" or VAR5="D"
or VAR6="A" or VAR6="B" or VAR6="C" or VAR6="D"
or VAR7="A" or VAR7="B" or VAR7="C" or VAR7="D"
or VAR8="A" or VAR8="B" or VAR8="C" or VAR8="C"
or VAR9="A" or VAR9="B" or VAR9="C" or VAR9="D"
or VAR10="A" or VAR10="B" or I10_D10="C" or VAR10="D"
or VAR12="A" or VAR12="B" or VAR12="C" or VAR12="D"
or VAR13="A" or VAR13="B" or VAR13="C" or VAR13="D"
or VAR14="A" or VAR14="B" or VAR14="C" or VAR14="D"
or VAR15="A" or VAR15="B" or VAR15="C" or VAR15="D"
or VAR6="A" or VAR16="B" or VAR16="C" or VAR16="D"
or VAR17="A" or VAR17="B" or VAR17="C" or VAR17="D"
or VAR18="A" or VAR18="B" or VAR18="C" or VAR18="C"
or VAR19="A" or VAR19="B" or VAR19="C" or I10_D19="D"
or VAR20="A" or VAR20="B" or I10_D20="C" or VAR20="D"
or VAR21="D" or VAR22="A" or VAR22="B" or VAR22="C" or VAR22="D"
or VAR23="A" or VAR23="B" or VAR23="C" or VAR23="D"
or VAR24="A" or VAR24="B" or VAR24="C" or VAR24="D"
or VAR25="A" or VAR25="B" or VAR25="C" or VAR25="D"
or VAR26="A" or VAR26="B" or VAR26="C" or VAR26="D"
or VAR27="A" or VAR27="B" or VAR27="C" or VAR27="D"
or VAR28="A" or VAR28="B" or VAR28="C" or VAR28="C"
or VAR29="A" or VAR29="B" or VAR29="C" or VAR29="D"
or VAR30="A" or VAR30="B" or I10_D30="C" or VAR30="D")
then Group=1; else Group=0;
You probably don't need a macro, however a macro might be faster.
%let value=a;
data want;
set have;
array var[50];
keepit=1;
do i=1 to 50;
keepit = keepit and (var[i]="&value");
if ^keepit then
leave;
end;
if keepit;
drop i keepit;
run;
I create a signal variable and update it's value, it will be false if any value in the var[] array is not the &value. I leave the loop early if we find 1 non-matching value, to make it more efficient.
It's not exactly clear what you want. If you want to avoid checking all variables you can use WHICHC to find if any in a list are A.
X = whichc('a', of var1-var30);
If you want to see what different groups you have across all the variables, I think a big proc freq is what you want:
proc freq data=have noprint;
table var1*var2*var3*var4....*var30*gender*age / list out=table_counts;
run;
And then check the table_counts data set to see if that has what you want.
If neither of these are what you want, you need to add more details to your question. A sample of data and expected output would be perfect.
When I need to search several variables for a particular value what I will do is - combine all variables into one string and then search that string. Like this:
*** CREATE TEST DATA ***;
data have;
infile cards;
input VAR1 $ VAR2 $ VAR3 $ VAR4 $ VAR5 $;
cards;
J J K A M
S U I O P
D D D D D
l m n o a
Q U J C S
;
run;
data want;
set have;
*** USE CATS FUNCTION TO CONCATENATE ALL VAR# INTO ONE VARIABLE ***;
allvar = cats(var1, var2, var3, var4, var5);
*** IF NEEDED, APPLY UPCASE TO CONCATENATED VARIABLE ***;
*allvar = upcase(allvar);
*** USE INDEXC FUNCTION TO SEARCH NEW VARIABLE ***;
if indexc(allvar, "ABCD") > 0 then group = 1;
else group = 0;
run;
I'm not sure if this is exactly what you need, but hopefully this is something you can modify for your particular task.
The code as posted is testing if ANY of a list of variables match ANY of a list of values.
Let's make a simple test dataset.
data have ;
input id (var1-var5) ($);
cards;
1 E F G H I
2 A B C D E
;;;;
Make one array of the values you want to find and one array of the variables you want to check. Loop over the list of variables until you either find one that contains one of the values or you run out of variables to test.
data want ;
set have;
array values (4) $8 _temporary_ ('A' 'B' 'C' 'D');
array vars var1-var5 ;
group=0;
do i=1 to dim(vars) until (group=1);
if vars(i) in values then group=1;
end;
drop i;
run;
You could avoid the array for the list of values if you want.
if vars(i) in ('A' 'B' 'C' 'D') then group=1;
But using the array will allow you to make the loop run over the list of values instead of the list of variables.
do i=1 to dim(values) until (group=1);
if values(i) in vars then group=1;
end;
Which might be important if you wanted to keep the variable i to indicate which value (or variable) was first matched.
I have a database with serveral variables, including one, RIF, that hase an x^2 shape relative to another variable, Y.
I want to obtain two seperate databases, separated based on whether the observation is on the decreasing or the increasing part of the curve.
I thought I had something by using the lag function, but my code does not work.
proc sort data=have; by y; run;
data want;
set have;
do while (rif<=lag(rif));
Part=1;
end;
if Part ne 1 then Part=2
run;
And the separating given Part, but it seems to create infintite loop.
Is there a mistake in my code / is there a better way of doing this
data have;
do x = -10 to 10 by 1;
y = x**2;
output;
end;
run;
data want;
set have;
lag_y = lag(y);
if _n_ = 1 then Part=.;
else if y <= lag_y then Part=1;
else Part=2;
drop lag_y;
run;
I have a SAS dataset as follow :
Key A B C D E
001 1 . 1 . 1
002 . 1 . 1 .
Other than keeping the existing varaibales, I want to replace variable value with the variable name if variable A has value 1 then new variable should have value A else blank.
Currently I am hardcoding the values, does anyone has a better solution?
The following should do the trick (the first dstep sets up the example):-
data test_data;
length key A B C D E 3;
format key z3.; ** Force leading zeroes for KEY;
key=001; A=1; B=.; C=1; D=.; E=1; output;
key=002; A=.; B=1; C=.; D=1; E=.; output;
proc sort;
by key;
run;
data results(drop = _: i);
set test_data(rename=(A=_A B=_B C=_C D=_D E=_E));
array from_vars[*] _:;
array to_vars[*] $1 A B C D E;
do i=1 to dim(from_vars);
to_vars[i] = ifc( from_vars[i], substr(vname(from_vars[i]),2), '');
end;
run;
It all looks a little awkward as we have to rename the original (assumed numeric) variables to then create same-named character variables that can hold values 'A', 'B', etc.
If your 'real' data has many more variables, the renaming can be laborious so you might find a double proc transpose more useful:-
proc transpose data = test_data out = test_data_tran;
by key;
proc transpose data = test_data_tran out = results2(drop = _:);
by key;
var _name_;
id _name_;
where col1;
run;
However, your variables will be in the wrong order on the output dataset and will be of length $8 rather than $1 which can be a waste of space. If either points are important (they rsldom are) and both can be remedied by following up with a length statement in a subsequent datastep:-
option varlenchk = nowarn;
data results2;
length A B C D E $1;
set results2;
run;
option varlenchk = warn;
This organises the variables in the right order and minimises their length. Still, you're now hard-coding your variable names which means you might as well have just stuck with the original array approach.