Insert more rows based on conditions from other rows - sas

Question

I have the following dataset:
data have;
input year firm_id location_id action action_amount operate new_entry
;
cards;
2013 28013 6085 1 10000 0 0
2015 28013 6085 1 12000 0 0
2015 28013 29189 1 10000 0 0
2016 28013 34019 1 5000 1 1
2017 28013 34019 0 0 1 2
2011 120609 9003 1 7000 0 0
2012 120609 9003 0 0 1 1
2013 120609 9003 1 5000 1 2
2012 247908 23001 1 9000 0 0
2013 247908 23001 1 8000 0 0
2014 247908 23001 1 8500 1 1
2015 247908 23001 0 0 1 2
2003 356123 1001 0 0 0 0
2004 356123 1001 0 0 0 0
2009 356123 1001 1 9800 1 1
;
run;
I want to add additional rows and two new variables called "pre_action" and "pre_action_amount" to obtain the following dataset:
data want;
input year firm_id location_id action action_amount operate new_entry pre_action pre_action_amount
;
cards;
2013 28013 6085 1 10000 0 0 . .
2014 28013 6085 0 0 0 0 1 10000
2015 28013 6085 1 12000 0 0 . .
2016 28013 6085 0 0 0 0 1 12000
2015 28013 29189 1 6500 0 0 . .
2016 28013 29189 0 0 0 0 1 6500
2016 28013 34019 1 5000 1 1 0 0
2017 28013 34019 0 0 1 2 . .
2011 120609 9003 1 7000 0 0 . .
2012 120609 9003 0 0 1 1 1 7000
2013 120609 9003 1 5000 1 2 . .
2012 247908 23001 1 9000 0 0 . .
2013 247908 23001 1 8000 0 0 1 9000
2014 247908 23001 1 8500 1 1 1 8000
2015 247908 23001 0 0 1 2 . .
2003 356123 1001 0 0 0 0 . .
2004 356123 1001 0 0 0 0 0 0
2005 356123 1001 0 0 0 0 0 0
2009 356123 1001 1 9800 1 1 0 0
;
run;
The rules are as follows:
1) First, consider only the rows with operate = 0.
For each firm_id and location_id pair, if in the following year there is no row with the same firm_id and location_id, then create a new row with the following year and same firm_id and location_id pair. The variables action, action_amount, operate, and new_entry are all set to 0, while pre_action and pre_action_amount is set to be the value of action and action_amount in the previous year. Example: In year 2013, for the firm_id/location_id pair 28013/6085, we have operate = 0. But in 2014, there are no observations for this firm_id/location_id pair. So we set action, action_amount, operate, and new_entry to be 0 and pre_action=1 and pre_action_amount=10000 which are the values for action and action_amount in 2013.
For each firm_id and location_id pair, if in the following year there is a row with the same firm_id and location_id, then simply set pre_action and pre_action_amount to be the value of action and action_amount in the previous year. Example: In year 2011 for firm_id/location_id 120609/9003, we have operate=0. But in the next year 2012, there is a row with this firm_id/location_id pair. So we set pre_action=1 and pre_action_amount=7000 which are the values for action and action_amount in 2011. Another example is in year 2003, for the firm_id/location_id 356123/1001.
2) Now consider the rows with new_entry=1 that do not yet have a value of pre_action and pre_action_amount. Set both pre_action and pre_action_amount to be 0.
3) All other values of pre_action and pre_action_amount are empty.
I am unsure of how to create these new rows given the complicated rules above, any help would be appreciated.

Answer 1

If your main problem is how to access information from the next record to determine if there is a gap in year (as there is no lead() function in SAS), following is an example that shows the use of the POINT= option of the SET statement to do precisely that.
The example is based on a slightly modified version of your data since:
it reduces the number of observations and variables in the dataset.
it uses just one variable to define the observation ID.
Example code:
/* Example that shows how to use the POINT= option of the SET statement
to look ahead at the next observation and decide whether there is a gap in YEAR. */
data example;
input year id action_amount;
cards;
2013 28013 10000
2015 28013 12000
2016 28013 5000
2017 28013 0
2003 356123 250
2004 356123 320
2009 356123 9800
;
run;
* Observations should be sorted by ID and YEAR for the next data step;
proc sort data=example; by id year; run;
* Process
data example_filled;
set example;
by id year;
* Create the variable that will be used in the POINT= option of the SET statement
* to retrieve information from the next observation;
_next_obs = _N_ + 1;
* Compute variables storing values from the previous observation;
retain pre_action_amount;
pre_action_amount = lag(action_amount);
if first.id then
call missing(pre_action_amount);
* Analyze possible YEAR gaps in a BY group and partly fill them;
if last.id then do;
*** No gaps to check => simply output the last observation of the BY group;
output;
end;
else do;
*** We should check for gaps in YEAR;
* 1) Send the current observation to the output dataset;
output;
* 2) Check if there is a gap > 1 based on the YEAR value from the next observation,
* and if so, fill in (just) ONE observation
* Note the use of the POINT= option of the SET statement to access info from next obs;
set example(keep=year rename=(year=next_year)) POINT=_next_obs;
if next_year > year + 1 then do;
* Compute pre action values as the action values of the obs just output above;
* All the other variables still maintain their values and will be carried over
* this newly created obs;
pre_action_amount = action_amount;
year = year + 1;
* Output the filled observation;
output;
end;
end;
drop next_year;
run;
Expected output dataset example_filled (the new filled observations are Obs = 2 and Obs = 8):
action_ pre_action_
Obs year id amount amount
1 2013 28013 10000 .
2 2014 28013 10000 10000
3 2015 28013 12000 10000
4 2016 28013 5000 12000
5 2017 28013 0 5000
6 2003 356123 250 .
7 2004 356123 320 250
8 2005 356123 320 320
9 2009 356123 9800 320
You may find handy to use this example as a basis to implement the logic you described in order to compute pre_action and pre_action_amount.
Note: here are other ways to access information from the next observation in a data step:
https://blogs.sas.com/content/sgf/2015/06/19/can-you-lag-and-lead-at-the-same-time-if-using-the-sas-data-step-yes-you-can/