Conditional transpose using SAS - sas

I have a dataset that looks like the following for multiple patients. I am trying to subtract each visit value from the baseline value of corresponding variables (which are sometimes missing).
Data Have:
Patient Variable Value Visit
A Height 100 Baseline
A Weight 50 Baseline
A HDCIRC 30 Baseline
A BMI 50 Baseline
A Height 100 a
A Weight 50 a
A HDCIRC 30 a
A BMI 50 a
A Height 100 b
A Weight 50 b
Data Want:
Patient Variable Value Visit BASELINE Change
A Height 100 Baseline 100 0
A Weight 50 Baseline 50 0
A HDCIRC 30 Baseline 30 0
A BMI 50 Baseline 50 0
A Height 120 a 100 20
A Weight 50 a 50 0
A HDCIRC 30 a 30 0
A BMI 34.7 a 50 -15.3
A Height 150 b 100 50
A Weight 51 b 50 1
My attempt would be to first create BASELINE and then calculate the change.
In order to get BASELINE, I've seen some people use a lag or a dif function. How can I correctly create the BASELINE variable?
proc sort data=have;
by patient visit;
;
data want;
set have;
by patient visit;
difstamp = dif(visit);
if first.patient then do;
dif=0;
end;
else dif=difstamp;
drop difstamp;
run;
proc sort data=want;
by timestamp;
run;

As alternative you could simply merge have with itself
data have;
input Patient $ Variable $ Value Visit $;
cards;
A Height 100 Baseline
A Weight 50 Baseline
A HDCIRC 30 Baseline
A BMI 50 Baseline
A Height 120 a
A Weight 50 a
A HDCIRC 30 a
A BMI 34.7 a
A Height 150 b
A Weight 51 b
;
proc sort;
by patient variable;
run;
data want;
merge have have(where=(__visit='Baseline') keep=patient variable value visit rename=(visit=__visit value=BASELINE))
;
by patient variable;
Change=Value-BASELINE;
drop __:;
run;

It probably helps to sort by PATIENT VARIABLE so that you can get the baseline.
If your VISIT variable doesn't properly sort BASELINE to the first visit then you can use WHERE= dataset options to make sure the baseline appear first.
data have;
input Patient $ Variable $ Value Visit $;
cards;
A Height 100 Baseline
A Weight 50 Baseline
A HDCIRC 30 Baseline
A BMI 50 Baseline
A Height 120 a
A Weight 50 a
A HDCIRC 30 a
A BMI 34.7 a
A Height 150 b
A Weight 51 b
;
proc sort;
by patient variable visit;
run;
data want;
set have(in=in1 where=(visit='Baseline'))
have(in=in2 where=(visit^='Baseline'))
;
by patient variable ;
if first.variable then do;
if in1 then baseline=Value;
else baseline=.;
retain baseline;
end;
if n(value,baseline)=2 then change=value-baseline;
run;
Result:
Obs PATIENT VARIABLE VALUE VISIT BASELINE CHANGE
1 A BMI 50.0 Baseline 50 0.0
2 A BMI 34.7 a 50 -15.3
3 A HDCIRC 30.0 Baseline 30 0.0
4 A HDCIRC 30.0 a 30 0.0
5 A Height 100.0 Baseline 100 0.0
6 A Height 120.0 a 100 20.0
7 A Height 150.0 b 100 50.0
8 A Weight 50.0 Baseline 50 0.0
9 A Weight 50.0 a 50 0.0
10 A Weight 51.0 b 50 1.0

Related

how SAS create an external report(.xlsx file) with filter summation

I have a data for sales in 3 months (sale1, sale2 and sale3), and I need to show the the different summations with different filters.
data sales;
input area load $ prod : $ sale1 sale2 sale3;
diff=sale3-sale2;
datalines;
1 Y p1 109 117 138
1 N p1 23 29 20
1 Y p2 78 70 68
1 N p2 63 19 22
2 Y p1 49 36 32
2 N p1 50 39 44
2 Y p3 138 157 158
2 N p3 110 126 107
3 Y p2 251 267 259
3 N p2 182 184 160
;
run;
ods excel close;
ods excel file="/C:/data/t1.xlsx"
options (sheet_name="tab1" frozen_headers='3' frozen_rowheaders='2'
embedded_footnotes='yes' autofilter='1-8');
proc report data=sales nocenter;
column area load prod sale1 sale2 sale3 diff change;
define area -- diff/ display;
define sale1-- diff / analysis sum format=comma12. style(column)=[cellwidth=.5in];
define change / computed format=percent8.2 '% change' style(column)=[cellwidth=.8in];
compute change;
change = diff.sum/sale2.sum;
if change >= 0.1 then call define ("change",'STYLE','STYLE=[color=red
fontweight=bold]');
if change <= -0.1 then call define ("change",'STYLE','STYLE=[color=blue
fontweight=bold]');
endcomp;
rbreak after / summarize style=[background=lightblue font_weight=bold];
run;
ods excel close;
this report with no filtering looks likeoriginal report
but if I filter with column load='Y' in the .xlsx file, i want to see the result like this:
output with filter
wonder if anyone can help, thanks!

How to select a percentage of values from a column in SAS?

I have 70 databases of different sizes (same number of columns, different numbers of lines).
I need to get the 25% higher values and the 25% lower values considering a given column VAR1.
I have:
id VAR1
1 10
2 -5
3 -12
4 7
5 12
6 7
7 -9
8 -24
9 0
10 6
11 -18
12 22
Sorting by VAR1, I need to select the rows (all columns) containing the 3 smallest and the 3 largest (25% from each extreme), i.e.,
id VAR1
8 -24
11 -18
3 -12
7 -9
2 -5
9 0
10 6
4 7
6 7
1 10
5 12
12 22
I need to keep in the database the rows (all columns) that contain the VAR1 equal to -24, -18, -12, 10, 12 and 22.
id VAR1
8 -24
11 -18
3 -12
1 10
5 12
12 22
What I’ve been thinking:
Order column VAR1 in ascending order;
Create a numbered column from 1 to N (n=_N_) - in this case, N=12;
I do a=N*0.25 (to have the value that represents 25%);
I do b=N-a (to have the value that represents the "last" 25%).
So, I can use keep:
if N<a.... I will have the first 25% (the smallest).
if N>b.... I will have the last 25% (the largest).
I can calculate a and b.
But I’m not getting the maximum value of N in this case 12.
I will repeat this for the 70 database, I would not like to have to enter this maximum value every time (it varies from one database to another).
I need help to "fix" the maximum value (N) without having to type it (even if it is repeated in all the lines of another "auxiliary column").
Or if there’s some better way to get those 25% from each end.
My code:
proc sort data=have; by VAR1; run;
data want; set have;
seq=_N_;
N=max(seq); *N=max. value of lines. (I stopped here and don’t know if below is right);
a=N*0.25;
b=N-b;
if N<a;
if N>b;
run;
Thank you very much!
Proc RANK computes percentiles that you can use to select the desired rows.
Example:
data have1 have2 have3 have4 have5;
do id = 1 to 100;
X = ceil(rand('normal', 0, 10));
if id < 60 then output have1;
if id < 70 then output have2;
if id < 80 then output have3;
if id < 90 then output have4;
if id < 100 then output have5;
end;
run;
proc rank data=have1 percent out=want1(where=(pct not between 25 and 75)) ;
var x;
ranks pct;
run;
proc rank data=have2 percent out=want2(where=(pct not between 25 and 75)) ;
var x;
ranks pct;
run;
proc rank data=have3 percent out=want3(where=(pct not between 25 and 75)) ;
var x;
ranks pct;
run;

SAS_Conditional Cumulative Sum

My question is about the conditional cumulative sum in SAS. I think it can be explained better by using sample. I have following dataset:
Date Value
01/01/2001 10
02/01/2001 20
03/01/2001 30
04/01/2001 15
05/01/2001 25
06/01/2001 35
07/01/2001 20
08/01/2001 45
09/01/2001 35
I want to find the cumulative sum of value. My condition is if cumulative sum more than 70, it should be 70 and the next cumulative sum should be began from the excessive value over 70 and so on.. More preciesly, my new data should be:
Date Value Cumulative
01/01/2001 10 10
02/01/2001 20 30
03/01/2001 30 60
04/01/2001 15 70
05/01/2001 25 30 ( 75-70=5+25=30)
06/01/2001 35 65
07/01/2001 20 70
08/01/2001 45 60 ( 85-70=15+45=60)
09/01/2001 35 95 ( because its last value)
Many thanks in advance
Here is a solution, although there is bound to be one more elegant. It's split into two parts with if eof to satisfy the last observation condition.
data want;
set test end = eof;
if eof ^= 1 then do;
if cumulative = 70 then cumulative = extra;
Cumulative + value;
extra = cumulative - 70;
if extra > 0 then do;
cumulative = 70;
end;
end;
retain extra;
retain cumulative;
if eof = 1 then cumulative + value;
run;

SAS: Filling the missing values by block of data

Say that I have the following database:
Min Rank Qty
2 1 100
2 2 90
2 3 80
2 4 70
5 1 110
5 2 100
5 3 90
5 4 80
5 5 70
7 1 120
7 2 110
7 3 100
7 4 90
I need to have the database with the continuous values for minutes like this:
Min Rank Qty
2 1 100
2 2 90
2 3 80
2 4 70
3 1 100
3 2 90
3 3 80
3 4 70
4 1 100
4 2 90
4 3 80
4 4 70
5 1 110
5 2 100
5 3 90
5 4 80
5 5 70
6 1 110
6 2 100
6 3 90
6 4 80
6 5 70
7 1 120
7 2 110
7 3 100
7 4 90
How can I do this in SAS? I just need to replicate the previous minute. The number of observations per minute varies...it can be 4 or 5 or more.
It is not that hard to imagine code that would do this, the problem is that it quickly starts to look messy.
If your dataset is not too large, one approach you could consider the following approach:
/* We find all gaps. the output dataset is a mapping: the data of which minute (reference_minute) do we need to create each minute of data*/
data MINUTE_MAPPING (keep=current_minute reference_minute);
set YOUR_DATA;
by min;
retain last_minute 2; *set to the first minute you have;
if _N_ NE 1 and first.min then do;
/* Find gaps, map them to the last minute of data we have*/
if last_minute+1 < min then do;
do current_minute=last_minute+1 to min-1;
reference_minute=last_minute;
output;
end;
end;
/* For the available data, we map the minute to itself*/
reference_minute=min;
current_minute=min;
output;
*update;
last_minute=min;
end;
run;
/* Now we apply our mapping to the data */
*you must use proc sql because it is a many-to-many join, data step merge would give a different outcome;
proc sql;
create table RESULT as
select YD.current_minute as min, YD.rank, YD.qty
MINUTE_MAPPING as MM
join YOUR_DATA as YD
on (MM.reference_minute=YD.min)
;
quit;
The more performant approach would involve trickery with arrays.
But i find this approach a bit more appealing (disclaimer: at first thought), it is quicker to grasp (disclaimer again: imho) for someone else afterwards.
For good measure, the array approach:
data RESULT (keep=min rank qty);
set YOUR_DATA;
by min;
retain last_minute; *assume that first record really is first minute;
array last_data{5} _TEMPORARY_;
if _N_ NE 1 and first.min and last_minute+1 < min then do; *gap found;
do current_min=last_minute+1 to min-1;
*store data of current record;
curr_min=min;
curr_rank=rank;
curr_qty=qty;
*produce records from array with last available data;
do iter=1 to 5;
min = current_minute;
rank = iter;
qty = last_data{iter};
if qty NE . then output; *to prevent output of 5th element where there are only 4;
end;
*put back values of actual current record before proceeding;
min=curr_min;
rank=curr_rank;
qty=curr_qty;
end;
*update;
last_minute=min;
end;
*insert data for use on later missing minutes;
last_data{rank}=qty;
if last.min and rank<5 then last_data{5}=.;
output; *output actual current data point;
run;
Hope it helps.
Note, currently no access to a SAS client where i am. So untested code, might contain a couple of typo's.
Unless you have an absurd number of observations, I think transposing would make this easy.
I don't have access to sas at the moment so bear with me (I can test it out tomorrow if you can't get it working).
proc transpose data=data out=data_wide prefix=obs_;
by minute;
id rank;
var qty;
run;
*sort backwards so you can use lag() to fill in the next minute;
proc sort data=data_wide;
by descending minute;
run;
data data_wide; set data_wide;
nextminute = lag(minute);
run;
proc sort data=data_wide;
by minute;
run;
*output until you get to the next minute;
data data_wide; set data_wide;
*ensure that the last observation is output;
if nextminute = . then output;
do until (minute ge nextminute);
output;
minute+1;
end;
run;
*then you probably want to reverse the transpose;
proc transpose data=data_wide(drop=nextminute)
out=data_narrow(rename=(col1=qty));
by minute;
var _numeric_;
run;
*clean up the observation number;
data data_narrow(drop=_NAME_); set data_narrow;
rank = substr(_NAME_,5)*1;
run;
Again, I can't test this now, but it should work.
Someone else may have a clever solution that makes it so you don't have to reverse-sort/lag/forward-sort. I feel like I have dealt with this before but the obvious solution for me right now is to have it sorted backwards at whatever prior sort you do (you can do the transpose with a descending sort no problem) to save you an extra sort.

Hash object in SAS - is it possible to merge two tables below using hash object?

Is it possible to merge below two tables using hash object in SAS 9.1 example below ? The main problemseems to be creation of Value variable w Result dataset. Problem is that each payment could pay for more than one charge, and sometimes more than one payment is need to pay for one charge and this tho cases could appear simultaneously. Does it problem has some general name ?
http://support.sas.com/rnd/base/datastep/dot/hash-getting-started.pdf
data TABLE1;
input ID_client ID_commodity Charge;
datalines;
1 111111111 100
1 222222222 200
2 333333333 300
2 444444444 400
2 555555555 500
;;;;
run;
data TABLE2;
input ID_client_hash ID_ofpayment paymentValue;
datalines;
1 11 50
1 12 50
1 13 100
1 14 50
1 15 50
2 21 500
2 22 200
2 23 100
2 24 200
2 25 200
;;;;
run;
data OUT;
input ID_client ID_commodity ID_ofpayment value;
datalines;
1 111111111 11 50
1 111111111 12 50
1 222222222 13 100
1 222222222 14 50
1 222222222 15 50
2 333333333 21 300
2 444444444 21 200
2 444444444 22 200
2 555555555 23 100
2 555555555 24 200
2 555555555 25 200
This might work for you - I have 9.2 and 9.2 has some significant hash improvements, but I think I behaved myself and only used what was there in 9.1. You might try crossposting this to SAS-L [SAS listserv] as Paul Dorfman (ie, The Hash Guru) reads that still I believe.
I assumed you want the 'leftovers' posted out. You may need to work on that part, if it's not working the way you want. This isn't terribly well tested, it works for your example dataset. I call missing the commodity for 24 and 25 since they're not used for that.
I'm pretty sure there's a more clean way to do the iteration than what I do, but since 9.2+ is what I use and we have multidata available, i've always used that instead of hash iterators so I don't know the cleaner methods.
data have;
input ID_client ID_commodity Charge;
datalines;
1 111111111 100
1 222222222 200
2 333333333 300
2 444444444 400
2 555555555 50
;;;;
run;
data for_hash;
input ID_client_hash ID_ofpayment paymentValue;
datalines;
1 11 50
1 12 50
1 13 100
1 14 50
1 15 50
2 21 500
2 22 200
2 23 100
2 24 200
2 25 200
;;;;
run;
data want;
*Create hash and hash iterator - must use iterator since 9.1 does not allow multidata option;
if _n_ = 1 then do;
format id_client_hash paymentValue id_ofpayment BEST12.;
declare hash h(dataset:'for_hash' , ordered: 'a');
h.defineKey('ID_client_hash','id_ofpayment'); *note I put id_client_hash, renaming the id - want to be able to compare them;
h.defineData('id_client_hash','id_ofpayment','paymentValue');
call missing(id_ofpayment,paymentValue, id_client_hash);
h.defineDone();
declare hiter hi('h');
end;
do _t = 1 by 1 until (last.id_client);
set have;
by id_client;
*Iterate through the hash and find the first record with the same ID_client;
do rc = hi.first() by 0 while (rc eq 0 and ID_client ne ID_client_hash);
rc = hi.next();
end;
*For the current charge record, iterate through the payment (hash) until all paid up.;
do while (charge gt 0 and rc eq 0 and ID_client=ID_client_hash);
if charge ge paymentValue then do; *If charge >= paymentvalue, use up the payment value;
value = paymentValue; *so whole paymentValue is value;
charge = charge - paymentValue; *charge is decremented by paymentValue;
output; *output row;
_id=ID_client_hash;
_pay=id_ofpayment;
rc = hi.next();
h.remove(key:_id,key:_pay); *remove payment row from hash now that it has been used up;
end;
else do; *this is if (remaining) charge is less than payment - we will not use all of the payment;
value = charge; *value is the remainder of the charge, ie, how much of payment was actually used;
paymentValue = paymentValue - charge; *paymentValue is the remainder of paymentValue;
charge= 0; *charge is zero now;
output; *output a row;
h.replace(); *replace paymentValue in the hash with the new value of paymentValue, minus charge;
end;
end; *end of iteration through hash - at this point, either charge = 0 or we have run out of payments with that ID;
if charge gt 0 then do;
value=-1*charge;
call missing(id_ofpayment);
output; *output a row for the charge, which is not paid;
end;
if last.id_client then do; *this is cleanup, checking to see if we have any leftover payments;
do while (rc=0); *iterate through the remaining hash;
do rc = hi.first() by 0 while (rc eq 0 and ID_client ne ID_client_hash);
rc = hi.next();
end;
if rc=0 then do;
call missing(id_commodity); *to make it clear this is a leftover payment;
value=paymentValue; *update the value;
output; *output the payment;
_id=ID_client_hash;
_pay=id_ofpayment;
rc = hi.next();
if rc= 0 then h.remove(key:_id,key:_pay); *remove the payment just output;
end;
end;
end;
end;
keep id_client id_ofpayment id_commodity value;
run;
Among other things, this isn't terribly fast - I do a lot of iterating that might be wasteful. It will be relatively faster if you don't have any payment ID_client records that aren't represented in the charge records- any that you do are getting skipped over, so that could end up super slow.
I'm not confident hash is the superior solution, at least pre-9.2; keyed UPDATE might be superior. UPDATE is pretty much made for transactional database structures, which this seems close to.