I need to generate a variable representing household income. I have each individual's income information and have identified the head (head == 1) and spouse (spouse == 1). Now I decide to define the household income as head's income plus their spouse's income if the head is married or the head's income if unmarried. In this case, I cannot simply use some command like
bys hhid : egen hhincome = total(income)
because there may be other members in the household who receiveincome, such as adult children living with their parents. So how can I achieve my goal, either using an egen function or other approach?
Thanks,Nick,it's really a tactful solution. Another a little bit lumpy solution come to my mind later on : Given I have used bysort prefix to generate the two dummy,head and spouse, the head and his/her spouse should appear as the first two members in each household group,then I can write something like bys hhid:gen hhincome=sum(income) in 1/2 if（pid==2&spouse==1）for married head and bys hhid:gen hhincome=income if(（pid==2&spouse==.）| hsize==1)for unmarried head or single-member household,where pidstands for individual's id and hsizeis a previously created dummy for household size
bysort hhid : egen hhincome = total(cond(head == 1, income, 0) + cond(spouse == 1, income, 0))
or
bysort hhid : egen hhincome = total((head == 1) * income + (spouse == 1) * income)
Related
I am trying to create a school-level learning inequality index based on the difference in learning outcomes between the top 10% and the bottom 50% of students in each school. The data are student-level (below is my attempt) but evidently, I am not creating the deciles at the school level. I imagine I need to use a foreach loop, but that is not yet something I know am fluent at.
My attempt:
bysort IDSCHOOL: egen pv__std_deciles = std(avg_pv)
xtile pv_deciles=avg_pv, nq(10)
bysort IDSCHOOL: egen school_pv_5 = mean(avg_pv) if pv_deciles<6
bysort IDSCHOOL: egen school_pv_10 = mean(avg_pv) if pv_deciles==10
egen max_sp_10=max(school_pv_10), by(IDSCHOOL)
egen max_sp_5=max(school_pv_5), by(IDSCHOOL)
gen school_pv_diff= max_sp_10 - max_sp_5
sort IDSCHOOL
quietly by IDSCHOOL: gen dup = cond(_N == 1, 0, _n)
drop if dup > 1
isid IDSCHOOL
I have a dataset with multiple subgroups (variable economist) and dates (variable temps99).
I want to run a tabsplit command that does not accept bysort or by prefixes. So I created a macro to apply my tabsplit command to each of my subgroups within my data.
For example:
levelsof economist, local(liste)
foreach gars of local liste {
display "`gars'"
tabsplit SubjectCategory if economist=="`gars'", p(;) sort
return list
replace nbcateco = r(r) if economist == "`gars'"
}
For each subgroup, Stata runs the tabsplit command and I use the variable nbcateco to store count results.
I did the same for the date so I can have the evolution of r(r) over time:
levelsof temps99, local(liste23)
foreach time of local liste23 {
display "`time'"
tabsplit SubjectCategory if temps99 == "`time'", p(;) sort
return list
replace nbcattime = r(r) if temps99 == "`time'"
}
Now I want to do it on each subgroups economist by date temps99. I tried multiple combination but I am not very good with macros (yet?).
What I want is to be able to have my r(r) for each of my subgroups over time.
Here's a solution that shows how to calculate the number of distinct publication categories within each by-group. This uses runby (from SSC). runby loops over each by-group, each time replacing the data in memory with the data from the current by-group. For each by-group, the commands contained in the user's program are executed. Whatever is left in memory when the user's program terminates is considered results and accumulates. Once all the groups have been processed, these results replace the data in memory.
I used the verbose option because I wanted to present the results for each by-group using nice formatting. The derivation of the list of distinct categories is done by splitting each list, converting to a long layout, and reducing to one observation per distinct value. The distinct_categories program generates one variable that contains the final count of distinct categories for the by-group.
* create a demontration dataset
* ------------------------------------------------------------------------------
clear all
set seed 12345
* Example generated by -dataex-. To install: ssc install dataex
clear
input str19 economist
"Carmen M. Reinhart"
"Janet Currie"
"Asli Demirguc-Kunt"
"Esther Duflo"
"Marianne Bertrand"
"Claudia Goldin"
"Bronwyn Hughes Hall"
"Serena Ng"
"Anne Case"
"Valerie Ann Ramey"
end
expand 20
bysort economist: gen temps99 = 1998 + _n
gen pubs = runiformint(1,10)
expand pubs
sort economist temps99
gen pubid = _n
local nep NEP-AGR NEP-CBA NEP-COM NEP-DEV NEP-DGE NEP-ECM NEP-EEC NEP-ENE ///
NEP-ENV NEP-HIS NEP-INO NEP-INT NEP-LAB NEP-MAC NEP-MIC NEP-MON ///
NEP-PBE NEP-TRA NEP-URE
gen SubjectCategory = ""
forvalues i=1/19 {
replace SubjectCategory = SubjectCategory + " " + word("`nep'",`i') ///
if runiform() < .1
}
replace SubjectCategory = subinstr(trim(SubjectCategory)," ",";",.)
leftalign // from SSC
* ------------------------------------------------------------------------------
program distinct_categories
dis _n _n _dup(80) "-"
dis as txt "fille = " as res economist[1] as txt _col(68) " temps = " as res temps99[1]
// if there are no subjects for the group, exit now to avoid a no obs error
qui count if !mi(trim(SubjectCategory))
if r(N) == 0 exit
// split categories, reshape to a long layout, and reduce to unique values
preserve
keep pubid SubjectCategory
quietly {
split SubjectCategory, parse(;) gen(cat)
reshape long cat, i(pubid)
bysort cat: keep if _n == 1
drop if mi(cat)
}
// show results and generate the wanted variable
list cat
local distinct = _N
dis _n as txt "distinct = " as res `distinct'
restore
gen wanted = `distinct'
end
runby distinct_categories, by(economist temps99) verbose
This is an example of the XY problem, I think. See http://xyproblem.info/
tabsplit is a command in the package tab_chi from SSC. I have no negative feelings about it, as I wrote it, but it seems quite unnecessary here.
You want to count categories in a string variable: semi-colons are your separators. So count semi-colons and add 1.
local SC SubjectCategory
gen NCategory = 1 + length(`SC') - length(subinstr(`SC', ";", "", .))
Then (e.g.) table or tabstat will let you explore further by groups of interest.
To see the counting idea, consider 3 categories with 2 semi-colons.
. display length("frog;toad;newt")
14
. display length(subinstr("frog;toad;newt", ";", "", .))
12
If we replace each semi-colon with an empty string, the change in length is the number of semi-colons deleted. Note that we don't have to change the variable to do this. Then add 1. See also this paper.
That said, a way to extend your approach might be
egen class = group(economist temps99), label
su class, meanonly
local nclass = r(N)
gen result = .
forval i = 1/`nclass' {
di "`: label (class) `i''"
tabsplit SubjectCategory if class == `i', p(;) sort
return list
replace result = r(r) if class == `i'
}
Using statsby would be even better. See also this FAQ.
I have observational data of observed matched pairs (supplier-buyer) which I would like to use to build a set of counterfactual observations.
Suppose we observe two suppliers (X and Y) and 3 buyers (A, B, and C).
We observe that supplier X has a $2M contract with buyer A, and a $5M contract with buyer B.
We also observe that supplier Y has only one contract with buyer C worth $4M.
I would like to build a set of counterfactuals based on the set of unmatched pairs. I want to place a hypothetical contract value for each counterfactual observation that is equal to the maximum of what each party contracted at in the observed data. For example, this is what we would get for the counterfactuals based on the observations above:
Counterfactual observation #1: X with C for a value of $5M (because X has $5M for his observed largest contract and C has $4M, $5M is higher)
Counterfactual observation #2: Y with A for a value of $4M (because Y has $4M for his observed largest contract and A has $2M, $4M is higher)
Counterfactual observation #3: Y with B for a value of $5M (because Y has $4M for his observed largest contract and B has $5M, $5M is higher)
and here's the Stata sample code depicting my data..
clear
input str3(supplier buyer) float cont_value
"X" "A" 2
"X" "B" 5
"Y" "C" 4
end
clear
set more off
input str3(supplier buyer) value
"X" "A" 2
"X" "B" 5
"Y" "C" 4
end
list
fillin supplier buyer
list, sepby(supplier)
bysort supplier : egen maxsupp = max(value)
bysort buyer : egen maxbuy = max(value)
egen value2 = rowmax(max*) if _fillin
replace value2 = value if !_fillin
sort supplier buyer
order value2, after(value)
list, sepby(supplier)
In addition to the answer on Statalist which merges a series of datasets, you can achieve this with fillin.
clear *
input str3(supplier buyer) float cont_value
"X" "A" 2
"X" "B" 5
"Y" "C" 4
end
fillin supplier buyer
list
tempvar cont_val cont_val1
by supplier, sort : egen `cont_val' = max(cont_value)
by buyer, sort : egen `cont_val1' = max(cont_value)
egen cont_val = rowmax(`cont_val' `cont_val1') if _fillin
replace cont_val = cont_value if !_fillin
drop _fillin
sort supplier buyer
list supplier buyer cont_value cont_val
I'm looking at the Current Population Survey in Stata, although this question could apply to any survey with individual weights.
It's straightforward to generate a table showing the mean of a variable -- say wages -- over time given individual weights:
table qtr [aw=pworwgt], contents(mean wage)
What I'd like to do automatically is show the average level of, in this example, wages, but with the proportions of certain categories fixed to a date.
So for example, let's say I have 6 educational categories (Less than HS, HS, Some College, AA, BA/BS, Grad School)... I'd want to see how wages would be different if I fixed the educational proportions of the workforce to their, say, 2005 levels.
Ideally, the solution would not be resource intensive for large-numbered categories. For example, I might want to do something similar with the CPS's detail occupational metric, which has hundreds of levels.
My gut tells me "margins" may be part of the solution but I'm not familiar enough with that command... also, I'd like to be able to generate table output so I can graph in other software.
ETA: Here's the way I tried to do this for fixing weights by age and sex: by cycling through all the data, comparing the contemporaneous proportions to the base quarter proportions, and then adjusting the individual weights accordingly. This takes a really long time to cycle through however.
local start = tq(1994q1)
local end = tq(2014q4)
local base = tq(2006q1)
tempvar pop2006
tempvar cohort2006
tempvar poptemp
gen pworwgt_a = pworwgt
levelsof pesex, local(sex)
sum pworwgt if qtr == `base'
gen `pop2006' = r(N)*r(mean)
gen `cohort2006' = .
gen `poptemp' = .
forvalues age = 16/85 {
foreach s in `sex' {
sum pworwgt if age == `age' & pesex == `s' & qtr == `base'
replace `cohort2006' = r(N)*r(mean)/`pop2006'
forvalues q = `start'/`end' {
sum pworwgt if qtr == `q'
replace `poptemp' = r(N)*r(mean)
sum pworwgt if age == `age' & pesex == `s' & qtr == `q'
replace pworwgt_a = pworwgt_a*`cohort2006'/((r(N)*r(mean))/`poptemp') if age == `age' & pesex == `s' & qtr == `q'
}
}
}
I don't have scope to test this, but here are suggested simplifications to the code segment. I don't address the main question, which I don't understand, partly because there is no precise description of data structure in the question.
To summarize suggestions:
Use summarize, meanonly when that is all you need and use r(sum) ditto.
Use scalars not variables for constants.
Shift repeated calculations to once-and-for-all calculations of variables. I think you can do even more of this, but I will stop here.
local start = tq(1994q1)
local end = tq(2014q4)
local base = tq(2006q1)
tempname pop2006 cohort2006
tempvar qassum qsum
// quarter-age-sex sums in a single variable
bysort qtr age pesex : gen double `qassum` = sum(pworwgt)
by qtr age pesex : replace `qassum` = `qassum`[_N]
// quarterly sums in a single variable
by qtr: gen double `qsum' = sum(pworwgt)
by qtr: replace `qsum` = `qsum'[_N]
gen pworwgt_a = pworwgt
levelsof pesex, local(sex)
sum pworwgt if qtr == `base', meanonly
scalar `pop2006' = r(sum)
forvalues age = 16/85 {
foreach s in `sex' {
sum pworwgt if age == `age' & pesex == `s' & qtr == `base', meanonly
scalar `cohort2006' = r(sum)/`pop2006'
replace pworwgt_a = pworwgt_a*`cohort2006'/`qassum'/`qsum' if age == `age' & pesex == `s'
}
}
I am trying to create a variable for each year in my data based on mathematical expressions of other variables (I have annual data and used "..." to avoid writing each year). I am using the summarize command in Stata to extract the standard deviation but Stata does not recognize the frac variable. I have tried to use egen but that results in an unknown function error. Using gen results in an already defined variable. I would appreciate anyone helping with the following code or pointing me to a link where this issue has been discussed.
foreach yr of numlist 1995...2012 {
local row = `yr' - 1994
local numerator = 100*(income - L1.income)
local denominator = ((abs(income) + abs(L1.income)) / 2)
local frac = (`numerator' / `denominator')
summarize frac
local sdfrac = r(sd)
matrix C[`row', 1] = `numerator'
matrix C[`row', 2] = `denominator'
matrix C[`row', 3] = `sdfrac'
}
If I am understanding your question right, maybe you don't need to use a loop until the end and then you can post the results to a postfile:
This is just a thought:
tempname memhold
tempfile filename
postfile `memhold' year sdfrac using `filename'
gen row=year-1994
gen numerator=100*(income-L1.income)
gen denominator=((abs(income)+abs(L1.income))/2)
gen frac=numerator/denominator
foreach yr of numlist 1995...2012 {
summarize frac if year=`yr'
local sdfrac=r(sd)
post `memhold' (year) (`sdfrac')
}
postclose `memhold'
clear all
use `filename'
*View Results
list
This code should get you a data set with the name of the year and the standard deviation of the frac variable as variables.
In a comment, OP added a question about code similar to this (but ignored the request to post it in a more civilised form). Note that backticks or left quotation marks in Stata clash with SO mark-up codes in comments. Presumably some
tempname memhold
definition preceded this.
postfile `memhold' year sdfrac sex race using myresults
levels of sex, local (s)
levelsof race, local (r)
foreach a of local s {
foreach b of local r {
forval yr = 1995/2012 {
summarize frac if year == `yr' & sex == `a' & race == `b'
post `memhold' (`yr') (`r(sd)') (`sex') (`race')
}
}
}
Let's focus on what the problem is. You want the standard deviations of frac for all combinations of sex, race and year in a separate file. That's one line
collapse (sd) frac, by(year sex race)
If you want to see a table alongside the data, consider
egen group = group(sex race year), label
and then
tab group, su(frac)
or
tabstat frac, by(group) stat(sd)
This code modifies that by #Pcarlitz, mostly by simplifying it. I can't check with your data, which I don't have.
It's too long to fit into a comment.
I would not use a temporary file as you want to save these results, it seems.
tempname memhold
postfile `memhold' year sdfrac using myresults
gen frac = (100*(income - L1.income))/((abs(income) + abs(L1.income))/2)
forval yr = 1995/2012 {
summarize frac if year==`yr'
post `memhold' (`yr') (`r(sd)')
}
postclose `memhold'
use myresults
list
UPDATE As in a later answer, consider collapse as a much simpler direct alternative here.