I am trying to match max daily data within a month to a monthly data.
data daily;
input permno $ date ret;
datalines;
1000 19860101 88
1000 19860102 90
1000 19860201 70
1000 19860202 55
1001 19860201 97
1001 19860202 74
1001 19860203 79
1002 19860301 55
1002 19860302 100
1002 19860301 10
;
run;
data monthly;
input permno $ date ret;
datalines;
1000 19860131 1
1000 19860228 2
1000 19860331 5
1001 19860331 3
1002 19860430 4
;
run;
The result I want is the following; (I want to match daily max data to one month lag monthly data. )
1000 19860102 90 1000 19860228 2
1000 19860201 70 1000 19860331 5
1001 19860201 97 1001 19860331 3
1002 19860302 100 1002 19860430 4
Below is what I have tried so far.
I want to have maximum ret value within a month so I have created yrmon to assign same yyyymm data for the same month daily data
data a1; set daily;
yrmon=year(date)*100 + month(date);
run;
In order to choose the maximum value(here, ret) within same yrmon group for the same permno, I used code below
proc means data=a1 noprint;
class permno yrmon ;
var ret;
output out= a2 max=maxret;
run;
However, it only got me permno yrmon ret data, leaving the original date data away.
data a3;
set a2;
new=intnx('month',yrmon,1);
format date new yymmn6.;
run;
But it won't work since yrmon is no longer date format.
Thank you in advance.
Hello
I am trying to match two different sets by permno(same company) but with one month lag (eg. daily9 dataset yrmon=198601 and monthly2 dataset yrmon=198602)
it is pretty difficult to handle for me because if I just add +1 in yrmon, 198612 +1 will not be 198701 and I am confused with handling these issues.
Can anyone help?
1) informat date1/date2 yymmn6. is used to read the date in yyyymm format
2) format date1/date2 yymmn6. is used to view the date in yyyymm format
3) intnx("months",b.date2,-1) is used to join the dates with lag of 1 month
data data1;
input date1 value1;
informat date1 yymmn6.;
format date1 yymmn6.;
cards;
200101 200
200212 300
200211 400
;
run;
data data2;
input date2 value2;
informat date2 yymmn6.;
format date2 yymmn6.;
cards;
200101 3000000
200102 4000000
200301 2000000
200212 2000000
;
run;
proc sql;
create table result as
select a.*,b.date2,b.value2 from
data1 a
left join
data2 b
on a.date1 = intnx("months",b.date2,-1);
quit;
My Output:
date1 |value1 |date2 |value2
200101 |200 |200102 |4000000
200211 |400 |200212 |2000000
200212 |300 |200301 |2000000
Let me know in case of any queries.
Related
I have data that looks like this:
data have;
format date monyy.;
input date:date9. group$ value1 value2;
datalines;
01JAN2020 A 100 10
01FEB2020 A 200 20
01JAN2020 B 300 30
01FEB2020 B 400 40
;
run;
But I need it to look like this, where value1 value2 are suffixed with date. This format is being used to export to Excel.
data want;
input group$ value1_JAN20 value1_FEB20 value2_JAN20 value2_FEB20;
datalines;
A 100 200 10 20
B 300 400 30 40
;
run;
I have tried PROC TRANSPOSE, but the actual values of value1 value2 are stored as date names with the names of value1 value2 stored in the column _NAME_.
Is there a way to do this with PROC TRANSPOSE, or is this only a data step/macro solution?
proc transpose data = have
out = try
delim = '_'n
name = date;
by group;
id date;
var value1 value2;
run;
Transpose twice. First to get a truly vertical structure. Then to generate the structure you want.
data have;
input date :date9. group $ value1 value2;
format date monyy.;
datalines;
01JAN2020 A 100 10
01FEB2020 A 200 20
01JAN2020 B 300 30
01FEB2020 B 400 40
;
proc transpose data=have out=tall;
by group date ;
var value1-value2;
run;
proc transpose data=tall out=want(drop=_name_) delim=_;
by group;
id _name_ date ;
var col1 ;
run;
Results:
value1_ value2_ value1_ value2_
Obs group JAN20 JAN20 FEB20 FEB20
1 A 100 10 200 20
2 B 300 30 400 40
I have the following data and used one of the existing answered questions to solve my data problem but could not get what I want. Here is what I have in my data
Amt1 is populated when the Evt_type is Fee
Amt2 is populated when the Evt_type is REF1/REF2
I don't want to display any observations after the last Flag='Y'
If there is no Flag='Y' then I want all the observations for that id (e.g. id=102)
I want to display if the next row for that id is a Fee followed by REF1/REF2 after flag='Y' (e.g. id=101) However I don't want if there is no REF1/REF2 (e.g.id=103)
Have:
id Date Evt_Type Flag Amt1 Amt2
101 2/2/2019 Fee 5
101 2/3/2019 REF1 Y 5
101 2/4/2019 Fee 10
101 2/6/2019 REF2 Y 10
101 2/7/2019 Fee 4
101 2/8/2019 REF1
102 2/2/2019 Fee 25
102 2/2/2019 REF1 N 25
103 2/3/2019 Fee 10
103 2/4/2019 REF1 Y 10
103 2/5/2019 Fee 10
Want:
id Date Evt_Type Flag Amt1 Amt2
101 2/2/2019 Fee 5
101 2/3/2019 REF1 Y 5
101 2/4/2019 Fee 10
101 2/6/2019 REF2 Y 10
101 2/7/2019 Fee 4
101 2/8/2019 REF1
102 2/2/2019 Fee 25
102 2/2/2019 REF1 N 25
103 2/4/2019 REF1 Y 10
103 2/5/2019 Fee 10
I tried the following
data want;
set have;
by id Date;
drop count;
if (first.id or first.date) and FLAG='Y' then
do;
retain count;
count=1;
output;
return;
end;
if count=1 and ((first.id or first.date) and Flag ne 'Y') then
do;
retain count;
delete;
return;
end;
output;
run;
Any help is appreciated.
Thanks
A technique known as DOW loop can perform a computation that measures a group in some way and then, in a second loop, apply that computation to members of the group.
The DOW relies on a SET statement inside the loop. In this case the computation is 'what row in the group is the last one having flag="Y".
data want;
* DOW loop, contains computation;
_max_n_with_Y = 1e12;
do _n_ = 1 by 1 until (last.id);
set have;
by id;
if flag='Y' then _max_n_with_Y = _n_;
end;
* Follow up loop, applies computation;
do _n_ = 1 to _n_;
set have;
if _n_ <= _max_n_with_Y then OUTPUT;
end;
drop _:;
run;
Here is one way
data have;
input id $ Date : mmddyy10. Evt_Type $ Flag $ Amt1 Amt2;
format Date mmddyy10.;
infile datalines dsd missover;
datalines;
101,2/2/2019,Fee,,5,
101,2/3/2019,REF1,Y,,5
101,2/4/2019,Fee,,10,
101,2/6/2019,REF2,Y,,10
101,2/7/2019,Fee,,4,
102,2/2/2019,Fee,,25,
102,2/2/2019,REF1,N,25,
;
data want;
do _N_ = 1 by 1 until (last.id);
set have;
by id;
if flag = "Y" then _iorc_ = _N_;
end;
do _N_ = 1 to _N_;
set have;
if _N_ le _iorc_ then output;
end;
_iorc_=1e7;
run;
During some data cleaning process, there is a need to compare the data between different rows. For example, if the rows have the same countryID and subjectID then keep the largest temperature:
CountryID SubjectID Temperature
1001 501 36
1001 501 38
1001 510 37
1013 501 36
1013 501 39
1095 532 36
In this case like this, I will use the lag() function as follows.
proc sort table;
by CountryID SubjectID descending Temperature;
run;
data table_laged;
set table;
CountryID_lag = lag(CountryID);
SubjectID_lag = lag(SubjectID);
Temperature_lag = lag(Temperature);
if CountryID = CountryID_lag and SubjectID = SubjectID_lag then do;
if Temperature < Temperature_lag then delete;
end;
drop CountryID_lag SubjectID_lag Temperature_lag;
run;
The code above may work.
But I still want to know if there are any better ways to solve this kind of questions?
I think you complicate task. You can use proc sql and max function:
proc sql noprint;
create table table_laged as
select CountryID,SubjectID,max(Temperature)
from table
group by CountryID,SubjectID;
quit;
I don't know if you want it that way but you code would keep the highest temperatures
So when you have 2 1 3 for one subject if will keep 3. But when you have 1 4 3 4 4 it will keep 4 4 4. Better is to keep simple the first row for each subject which is the highest because of descending order.
proc sort data = table;
by CountryID SubjectID descending Temperature;
run;
data table_laged;
set table;
by CountryID SubjectID;
if first.SubjectID;
run;
You can use double DOW technique to:
Compute a measure over a group,
Apply the measure to items in the group.
The benefit of DOW looping is a single pass over the data set when incoming data is already grouped.
In this question, 1. is to identify the row in the group with the first highest temperature, and 2. is to select the row for output.
data want;
do _n_ = 1 by 1 until (last.SubjectId);
set have;
by CountryId SubjectId;
if temperature > _max_temp then do;
_max_temp = temperature;
_max_at_n = _n_;
end;
end;
do _n_ = 1 to _n_;
set have;
if _n_ = _max_at_n then OUTPUT;
end;
drop _:;
run;
The traditional procedural technique is Proc MEANS
data have;input
CountryID SubjectID Temperature; datalines;
1001 501 36
1001 501 38
1001 510 37
1013 501 36
1013 501 39
1095 532 36
run;
proc means noprint data=have;
by countryid subjectid;
output out=want(drop=_:) max(temperature)=temperature;
run;
If the data is disordered in CountryID and SubjectID going into the data step, a hash object can be used or SQL per #Aurieli.
I am working on a dataset in SAS to get the next observation's score should be the current observation's value for the column Next_Row_score. If there is no next observation then the current observation's value for the column Next_Row_score should be 'null'per group(ID). For better illustration i have provided the sample below dataset :
ID Score
10 1000
10 1500
10 2000
20 3000
20 4000
30 2500
Resultant output should be like -
ID Salary Next_Row_Salary
10 1000 1500
10 1500 2000
10 2000 .
20 3000 4000
20 4000 .
30 2500 2500
Thank you in advance for your help.
data want(drop=_: flag);
merge have have(firstobs=2 rename=(ID=_ID Score=_Score));
if ID=_ID then do;
Next_Row_Salary=_Score;
flag+1;
end;
else if ID^=_ID and flag>=1 then do;
Next_Row_Salary=.;
flag=.;
end;
else Next_Row_Salary=score;
run;
Try this :
data have;
input ID Score;
datalines;
10 1000
10 1500
10 2000
20 3000
20 4000
30 2500
;
run;
proc sql noprint;
select count(*) into :obsHave
from have;
quit;
data want2(rename=(id1=ID Score1=Salary) drop=ID id2 Score);
do i=1 to &obsHave;
set have point=i;
id1=ID;
Score1=Score;
j=i+1;
set have point=j;
id2=ID;
if id1=id2 then do;
Next_Row_Salary = Score;
end;
else Next_Row_Salary=".";
output;
end;
stop;
;
run;
There is a simpler (in my mind, at least) proc sql approach that doesn't involve loops:
data have;
input ID Score;
datalines;
10 1000
10 1500
10 2000
20 3000
20 4000
30 2500
;
run;
/*count each observation's place in its ID group*/
data have2;
set have;
count + 1;
by id;
if first.id then count = 1;
run;
/*if there is only one ID in a group, keep original score, else lag by 1*/
proc sql;
create table want as select distinct
a.id, a.score,
case when max(a.count) = 1 then a.score else b.score end as score2
from have2 as a
left join have2 (where = (count > 1)) as b
on a.id = b.id and a.count = b.count - 1
group by a.id;
quit;
I am trying to do a count on the number of births. the data looks this way
ID date
101 2016-01-01
101 2016-02-01
101 2016-02-01
102 2015-03-02
102 2016-04-01
103 2016-02-08
So now i want to create a count based on the date
the output expected is this way
ID date count
101 2016-01-01 1
101 2016-02-01 2
101 2016-02-01 2
102 2015-03-02 1
102 2016-04-01 2
103 2016-02-08 1
I am trying to do it by first and last and also the count from proc sql but I am missing something here.
data temp;
set temp;
by ID DATE notsorted;
if first.date then c=1;
else c+1;
if first.ID then m=1;
else m+1;
run;
Another solution with your original approach
data x;
input id : 3. date : ddmmyy10.;
FORMAT DATE ddmmyy10.;
datalines;
101 01-01-2016
101 02-01-2016
101 02-01-2016
102 03-02-2015
102 04-01-2016
103 02-08-2016
;
run;
data x;
set x;
by ID DATE notsorted;
if first.ID then c=0; /*reset count every time id changes*/
if first.date then c+1; /*raise count when date changes*/
run;
produces
Do you absolutely require to use first?
I would use proc freq to achieve this
data have;
infile datalines delimiter='09'x;
input ID $ date $10. ;
datalines;
101 2016-01-01
101 2016-02-01
101 2016-02-01
102 2015-03-02
102 2016-04-01
103 2016-02-08
;run;
proc freq DATA=have NOPRINT;
TABLES ID * date / OUT=want(drop=percent);
run;
creates this:
ID date count
101 2016-01-01 1
101 2016-02-01 2
102 2015-03-02 1
102 2016-04-01 1
103 2016-02-08 1
If you want to reproduce COUNT in the datastep you will have to use the double DOW. The dataset is SET twice. First time to count rows by ID and date. Second time to output all rows.
data out;
do _n_ = 1 by 1 until (last.date);
set test ;
by ID date;
if first.date then count = 1;
else count + 1;
end;
do _n_ = 1 by 1 until (last.date);
set test ;
by ID date;
output;
end;
run;
You forget to add RETAIN statement in your data-step.
data temp;
set temp;
retain c m 0;
by ID DATE notsorted;
if first.date then c=1;
else c+1;
if first.ID then m=1;
else m+1;
run;
Okay, I have edited the previous code. Hopefully this will suit your needs. Just make sure your date variable is in numeric or calendar format so that you can sort your table by ID and date first.
data want;
set have;
by id date;
if first.date then count=0;
count+1;
run;