| month | year | amount|
|-------|--------|-------|
| 1 | 2010 | 26 |
| 1 | 2010 | 26 |
| 2 | 2010 | 30 |
| 3 | 2010 | 35 |
| 3 | 2010 | 35 |
I need to figure out how to make another variable, that takes the prior month amount _n-1 and _n and divide it by 2, kind of like a moving average. The problem is that I need to do it by month and year, since there are multiples of the same month and year. There are other variables as well that are irrelevant, but that is why I can't just delete duplicates.
For example, for observation 5, I would need it to be (35+30+26) / 3
Your prescription and your example don't match at all. Your example is a mean of 3 monthly means, this month and the two previous. Your prescription is a month and the month previous.
Here is some technique that focuses on two possible meanings of your prescription.
* Example generated by -dataex-. For more info, type help dataex
clear
input byte month int year byte amount
1 2010 26
1 2010 26
2 2010 30
3 2010 35
3 2010 35
end
gen mdate = ym(year, month)
format mdate %tm
foreach w in total mean count {
egen `w' = `w'(amount), by(mdate)
}
gen wanted1 = (mean + mean[_n-1]) / 2 if mdate == mdate[_n-1] + 1
bysort mdate (wanted1) : replace wanted1 = wanted1[_n-1] if missing(wanted1)
gen wanted2 = (total + total[_n-1]) / (count + count[_n-1]) if mdate == mdate[_n-1] + 1
bysort mdate (wanted2) : replace wanted2 = wanted2[_n-1] if missing(wanted2)
list, sepby(mdate)
+----------------------------------------------------------------------------+
| month year amount mdate total mean count wanted1 wanted2 |
|----------------------------------------------------------------------------|
1. | 1 2010 26 2010m1 52 26 2 . . |
2. | 1 2010 26 2010m1 52 26 2 . . |
|----------------------------------------------------------------------------|
3. | 2 2010 30 2010m2 30 30 1 28 27.33333 |
|----------------------------------------------------------------------------|
4. | 3 2010 35 2010m3 70 35 2 32.5 33.33333 |
5. | 3 2010 35 2010m3 70 35 2 32.5 33.33333 |
+----------------------------------------------------------------------------+
Related
I am trying to recode a variable that indicates total number of responses to a multiple response survey question. Question 4 has options 1, 2, 3, 4, 5, 6, and participants may choose one or more options when submitting a response. The data is currently coded as binary outputs for each option: var Q4___1 = yes or no (1/0), var Q4___2 = yes or no (1/0), and so forth.
This is the tabstat of all yes (1) responses to the 6 Q4___* variables
Variable | Sum
-------------+----------
q4___1 | 63
q4___2 | 33
q4___3 | 7
q4___4 | 2
q4___5 | 3
q4___6 | 7
------------------------
total = 115
I would like to create a new variable that encapsulates these values.
Can someone help me figure out how to create this variable, and if coding a variable in this manner for a multiple option survey question is valid?
When I used the replace command the total number of responses were not adding up, as shown below
gen q4=.
replace q4 =1 if q4___1 == 1
replace q4 =2 if q4___2 == 1
replace q4 =3 if q4___3 == 1
replace q4 =4 if q4___4 == 1
replace q4 =5 if q4___5 == 1
replace q4 =6 if q4___6 == 1
label values q4 primarysource`
q4 | Freq. Percent Cum.
------------+-----------------------------------
1 | 46 48.94 48.94
2 | 31 32.98 81.91
3 | 6 6.38 88.30
4 | 1 1.06 89.36
5 | 3 3.19 92.55
6 | 7 7.45 100.00
------------+-----------------------------------
Total | 94 100.00
UPDATE
to specify I am trying to create a new variable that captures the column sum of each question, not the rowtotal across all questions. I know that 63 participants responded yes to question 4 a) and 33 to question 4 b) so I want my new variable to reflect that.
This is what I want my new variable's values to look like.
q4
-------------+----------
q4___1 | 63
q4___2 | 33
q4___3 | 7
q4___4 | 2
q4___5 | 3
q4___6 | 7
------------------------
total = 115
The fallacy here is ignoring the possibility of multiple 1s as answers to the various Q4???? variables. For example if someone answers 1 1 1 1 1 1 to all questions, they appear in your final variable only in respect of their answer to the 6th question. Otherwise put, your code overwrites and so ignores all positive answers before the last positive answer.
What is likely to be more useful are
(1) the total across all 6 questions which is just
egen Q4_total = rowtotal(Q4????)
where the 4 instances of ? mean that by eye I count 3 underscores and 1 numeral.
(2) a concatenation of responses that is just
egen Q4_concat = concat(Q4????)
(3) a variable that is a concatenation of questions with positive responses, so 246 if those questions were answered 1 and the others were answered 0.
gen Q4_pos = ""
forval j = 1/6 {
replace Q4_pos = Q4_pos + "`j'" if Q4____`j' == 1
}
EDIT
Here is a test script giving concrete examples.
clear
set obs 6
forval j = 1/6 {
gen Q`j' = _n <= `j'
}
list
egen rowtotal = rowtotal(Q?)
su rowtotal, meanonly
di r(sum)
* install from tab_chi on SSC
tabm Q?
Results:
. list
+-----------------------------+
| Q1 Q2 Q3 Q4 Q5 Q6 |
|-----------------------------|
1. | 1 1 1 1 1 1 |
2. | 0 1 1 1 1 1 |
3. | 0 0 1 1 1 1 |
4. | 0 0 0 1 1 1 |
5. | 0 0 0 0 1 1 |
|-----------------------------|
6. | 0 0 0 0 0 1 |
+-----------------------------+
. egen rowtotal = rowtotal(Q?)
. su rowtotal, meanonly
. di r(sum)
21
. tabm Q?
| values
variable | 0 1 | Total
-----------+----------------------+----------
Q1 | 5 1 | 6
Q2 | 4 2 | 6
Q3 | 3 3 | 6
Q4 | 2 4 | 6
Q5 | 1 5 | 6
Q6 | 0 6 | 6
-----------+----------------------+----------
Total | 15 21 | 36
I have data that look like this:
| Country | Year | Firm | Profit |
|---------|------|------|--------|
| A | 1 | 1 | 10 |
| A | 1 | 2 | 20 |
| A | 1 | 3 | 30 |
| A | 1 | 4 | 40 |
I want to create a new variable for each firm i that calculates the following:
For example, the value of the variable for firm 1 would be:
max(20 - 10, 0) + max(30 - 10, 0) + max(40 - 10, 0)
How can I do this in Stata by country and year?
Below is a direct solution to your problem (note the use of dataex for providing example data):
* Example generated by -dataex-. To install: ssc install dataex
clear
input str1 Country float(Year Firm Profit)
"A" 1 1 10
"A" 1 2 20
"A" 1 3 30
"A" 1 4 40
end
generate Wanted = -Profit
bysort Country Year (Wanted): replace Wanted = sum(Profit) - _n * Profit
list
+-----------------------------------------+
| Country Year Firm Profit Wanted |
|-----------------------------------------|
1. | A 1 4 40 0 |
2. | A 1 3 30 10 |
3. | A 1 2 20 30 |
4. | A 1 1 10 60 |
+-----------------------------------------+
The logic behind it is the following:
Note: This was the first answer posted. It didn't avoid the pitfall of taking the OP's algebra literally and wanting to implement the calculation in terms of maxima within groups. But I realised after posting that there must be a much simpler way of doing it and #Romalpa Akzo got there, which is excellent. I undeleted this on request because it does show some machinery for looping over groups and implementing a calculation for each group with a customised Mata function.
Here I write a Mata function to return the wanted result for a group and then loop over the groups to populate a pre-defined variable.
To test the code for a dataset with more than one group, I use mpg from Stata's auto toy dataset.
mata :
void wanted (string scalar varname, string scalar usename, string scalar resultname) {
real scalar i
real colvector x, result, zero
result = x = st_data(., varname, usename)
zero = J(rows(x), 1, 0)
for(i = 1; i <= rows(x); i++) {
result[i] = sum(rowmax((x :- x[i], zero)))
}
st_store(., resultname, usename, result)
}
end
sysuse auto, clear
sort foreign rep78 mpg
egen group = group(foreign rep78), label
summarize group, meanonly
local G = r(max)
generate wanted = .
generate touse = 0
quietly forvalues g = 1 / `G' {
replace touse = group == `g'
mata : wanted("mpg", "touse", "wanted")
}
How did that work out? Here are some results:
. list mpg wanted group if foreign, sepby(group)
+--------------------------+
| mpg wanted group |
|--------------------------|
53. | 21 7 Foreign 3 |
54. | 23 3 Foreign 3 |
55. | 26 0 Foreign 3 |
|--------------------------|
56. | 21 35 Foreign 4 |
57. | 23 19 Foreign 4 |
58. | 23 19 Foreign 4 |
59. | 24 13 Foreign 4 |
60. | 25 8 Foreign 4 |
61. | 25 8 Foreign 4 |
62. | 25 8 Foreign 4 |
63. | 28 2 Foreign 4 |
64. | 30 0 Foreign 4 |
|--------------------------|
65. | 17 84 Foreign 5 |
66. | 17 84 Foreign 5 |
67. | 18 77 Foreign 5 |
68. | 18 77 Foreign 5 |
69. | 25 42 Foreign 5 |
70. | 31 18 Foreign 5 |
71. | 35 6 Foreign 5 |
72. | 35 6 Foreign 5 |
73. | 41 0 Foreign 5 |
|--------------------------|
74. | 14 . . |
+--------------------------+
So, how would that be applied to your data?
clear
input str1 Country Year Firm Profit
A 1 1 10
A 1 2 20
A 1 3 30
A 1 4 40
end
egen group = group(Country Year), label
summarize group, meanonly
local G = r(max)
generate wanted = .
generate touse = 0
quietly forvalues g = 1/`G' {
replace touse = group == `g'
mata: wanted("Profit", "touse", "wanted")
}
Results:
. list Firm Profit wanted, sepby(group)
+------------------------+
| Firm Profit wanted |
|------------------------|
1. | 1 10 60 |
2. | 2 20 30 |
3. | 3 30 10 |
4. | 4 40 0 |
+------------------------+
I am trying to create instruments from a three-dimensional panel dataset, as included below:
input firm year market price comp_avg
1 2000 10 1 .
3 2000 10 2 .
3 2001 10 3 .
1 2002 10 4 .
3 2002 10 5 .
1 2000 20 6 .
3 2000 20 7 .
1 2001 20 8 .
2 2001 20 9 .
3 2001 20 10 .
1 2002 20 20 .
2 2002 20 30 .
3 2002 20 40 .
2 2000 30 50 .
1 2001 30 60 .
2 2001 30 70 .
1 2002 30 80 .
2 2002 30 90 .
end
The instrument I am trying to create is the lagged (year-1) average price of a firm's competitors (those in the same market) in each market the firm operates in in a given year.
At the moment, I have some code that does the job, but I am hoping that I am missing something and can do this in a more clear or efficient way.
Here is the code:
// for each firm
qui levelsof firm, local(firms)
qui foreach f in `firms' {
// find all years for that firm
levelsof year if firm == `f', local(years)
foreach y in `years' {
// skip first year (because there is no lagged data)
if `y' == 2000 {
continue
}
// find all markets in that year
levelsof market if firm == `f' & year == `y', local(mkts)
local L1 = `y'-1
foreach m in `mkts' {
// get average of all compeitors in that market in the year prior
gen temp = firm != `f' & year == `L1' & market == `m'
su price if temp
replace comp_avg = r(mean) if firm == `f' & market == `m' & year == `y'
drop temp
}
}
}
The data I am working with are reasonably large (~1 million obs) so the faster the better.
clear
input firm year market price
1 2000 10 1
3 2000 10 2
3 2001 10 3
1 2002 10 4
3 2002 10 5
1 2000 20 6
3 2000 20 7
1 2001 20 8
2 2001 20 9
3 2001 20 10
1 2002 20 20
2 2002 20 30
3 2002 20 40
2 2000 30 50
1 2001 30 60
2 2001 30 70
1 2002 30 80
2 2002 30 90
end
bysort firm market (year) : gen Lprice = price[_n-1] if year - year[_n-1] == 1
bysort market year : egen total = total(Lprice)
bysort market year : egen count = count(Lprice)
gen mean_others = (total - cond(missing(Lprice), 0, Lprice)) ///
/ (count - cond(missing(Lprice), 0, 1))
sort market year
list market year firm price Lprice mean_others total count, sepby(market year)
+--------------------------------------------------------------------------+
| market year firm price Lprice price mean_o~s total count |
|--------------------------------------------------------------------------|
1. | 10 2000 1 1 . 1 . 0 0 |
2. | 10 2000 3 2 . 2 . 0 0 |
|--------------------------------------------------------------------------|
3. | 10 2001 3 3 2 3 . 2 1 |
|--------------------------------------------------------------------------|
4. | 10 2002 1 4 . 4 3 3 1 |
5. | 10 2002 3 5 3 5 . 3 1 |
|--------------------------------------------------------------------------|
6. | 20 2000 3 7 . 7 . 0 0 |
7. | 20 2000 1 6 . 6 . 0 0 |
|--------------------------------------------------------------------------|
8. | 20 2001 2 9 . 9 6.5 13 2 |
9. | 20 2001 3 10 7 10 6 13 2 |
10. | 20 2001 1 8 6 8 7 13 2 |
|--------------------------------------------------------------------------|
11. | 20 2002 1 20 8 20 9.5 27 3 |
12. | 20 2002 3 40 10 40 8.5 27 3 |
13. | 20 2002 2 30 9 30 9 27 3 |
|--------------------------------------------------------------------------|
14. | 30 2000 2 50 . 50 . 0 0 |
|--------------------------------------------------------------------------|
15. | 30 2001 2 70 50 70 . 50 1 |
16. | 30 2001 1 60 . 60 50 50 1 |
|--------------------------------------------------------------------------|
17. | 30 2002 2 90 70 90 60 130 2 |
18. | 30 2002 1 80 60 80 70 130 2 |
+--------------------------------------------------------------------------+
My approach breaks it down:
Calculate the previous price for the same firm and market. (#1 could also be done by declaring a (firm, market) pair a panel.)
The mean of other values (here previous prices) in the same market and year is the (sum of others MINUS this price) divided by (number of others MINUS 1).
#2 needs a modification as if this price is missing, you need to subtract 0 from both numerator and denominator. Stata's normal rules would render sum MINUS missing as missing, but this firm's previous price might be unknown, yet others in the same market might have known prices.
Note: There are small ways of speeding up your code, but this should be faster (so long as it is correct).
EDIT: Another solution (2 lines) using rangestat (must be installed using ssc inst rangestat):
bysort firm market (year) : gen Lprice = price[_n-1] if year - year[_n-1] == 1
rangestat Lprice, interval(year 0 0) by(market) excludeself
I have an unbalanced daily panel where entries occur at uneven times. I would like to generate the rolling sum of some variable x over the past 365 days. I can think of two ways to do this, but the first is memory hungry and the second is processor hungry. Is there a third alternative that avoids these problems?
Here are my two solutions. Is there a third solution without memory or speed problems?
clear
set obs 200
set seed 2001
/* panel variables */
generate id = 1 + int(2*runiform())
generate time = mdy(1, 1, 2000) + int(10*365*runiform())
format time %td
duplicates drop
xtset id time
/* data */
generate x = runiform()
/* first approach is to fill the panel with `tsfill` */
/* then remove "seasonality" with `s.` */
tsfill
generate sx = sum(x)
generate ssx = s365.sx
/* second approach without `tsfill` */
/* but nested loop is fairly slow */
drop if missing(x)
generate double ssx_alt = 0
forvalues i = 1/`= _N' {
local j = `i'
local delta = time[`i'] - time[`j']
while ((`j' > 0) & (`delta' < 365) & (id[`i'] == id[`j'])) {
local x = cond(missing(x[`j']), 0, x[`j'])
replace ssx_alt = ssx_alt + `x' in `i'
local j = `j' - 1
local delta = time[`i'] - time[`j']
}
}
The sum over the last # days is the difference between two cumulative sums, the cumulative sum to now and the cumulative sum to # days ago. The extension to panel data is easy, but not shown here. I don't think gaps disturb this principle once you have applied tsfill.
. set obs 20
obs was 0, now 20
. gen t = _n
. gen y = 100 + _n
. gen sumy = sum(y)
. tsset t
time variable: t, 1 to 20
delta: 1 unit
. gen diff = sumy - L10.sumy
(10 missing values generated)
. l
+------------------------+
| t y sumy diff |
|------------------------|
1. | 1 101 101 . |
2. | 2 102 203 . |
3. | 3 103 306 . |
4. | 4 104 410 . |
5. | 5 105 515 . |
|------------------------|
6. | 6 106 621 . |
7. | 7 107 728 . |
8. | 8 108 836 . |
9. | 9 109 945 . |
10. | 10 110 1055 . |
|------------------------|
11. | 11 111 1166 1065 |
12. | 12 112 1278 1075 |
13. | 13 113 1391 1085 |
14. | 14 114 1505 1095 |
15. | 15 115 1620 1105 |
|------------------------|
16. | 16 116 1736 1115 |
17. | 17 117 1853 1125 |
18. | 18 118 1971 1135 |
19. | 19 119 2090 1145 |
20. | 20 120 2210 1155 |
+------------------------+
I am doing reports with proc tabulate, but unable to add total in a report.
Example
+--------+------+----------+--------+---+---+---+
| Shop | Year | Month | Family | A | B | C |
+--------+------+----------+--------+---+---+---+
| raoas | 2006 | january | TA12 | 5 | 6 | 0 |
| taba | 2008 | january | TS01 | 0 | 1 | 1 |
| suptop | 2008 | april | TZ05 | 0 | 0 | 1 |
| taba | 2006 | December | TA12 | 5 | 6 | 0 |
| raoas | 2008 | january | TA15 | 0 | 2 | 0 |
| sup | 2008 | april | TQ05 | 0 | 1 | 1 |
+--------+------+----------+--------+---+---+---+
code
proc tabulate data=REPORTDATA_T6 format=12.;
CLASS YEAR;
var A C;
table (A C)*SUM='',YEAR=''
/box = 'YEAR';
TITLE 'FORECAST SUMMARY';
run;
output
YEAR 2006 2008 2009
A 800 766 813
C 854 832 812
I tried with... table(A C)*sum,year all... it will sum up for all the years but I want by year.
I tried with all the possible ways and tried... table(A C)*sum all,year. It will give number of observations ie N.. Thanx JON CLEMENTS But I dont want to add as TOTAL VARIABLE in the table, becoz this is a sample data but the number of variables are more then 10, some time I need to change variables, So, every time i dont want to add new variable as total.
I'm not sure if it's possible to do what you want in one step using only original data. Keyword ALL works only for summing up categories of CLASS-variables, but you want to sum up two different variables.
But it's easy enough with interim step, creating dataset where A, B, C variables will become categories of one variable:
data REPORTDATA_T6;
input Shop $ Year Month $ Family $ A B C;
datalines;
raoas 2006 january TA12 5 6 0
taba 2008 january TS01 0 1 1
suptop 2008 april TZ05 0 0 1
taba 2006 December TA12 5 6 0
raoas 2008 january TA15 0 2 0
sup 2008 april TQ05 0 1 1
;
run;
proc sort data=REPORTDATA_T6; by Shop Year Month Family; run;
proc transpose data=REPORTDATA_T6 out=REPORTDATA_T6_long;
var A B C;
by Shop Year Month Family;
run;
proc tabulate data=REPORTDATA_T6_long;
class _NAME_ YEAR;
var COL1;
table (_NAME_ all)*COL1=' '*SUM=' ', YEAR=' '
/box = 'YEAR';
TITLE 'FORECAST SUMMARY';
run;