I am trying to calculate the 95% binomial Wilson confidence interval for the proportion of people completing treatment by year (dataset is line-listed for each person).
I want to store the results into a matrix so that I can use the putexcel command to export the results to an existing Excel spreadsheet without changing the formatting of the sheet. I have created a binary variable dscomplete_binary which is 0 for a person if treatment was not completed, and 1 if treatment was completed.
I have tried the following:
bysort year: ci dscomplete_binary, binomial wilson level(95)
This gives output of each year with the 95% confidence intervals. Previously I used statsby to collapse the dataset to store the results in variables but this clears the dataset from the memory and so I have to constantly re-open it.
Is there a way to run the command and store the results in a tabular format so that the data is stored in a similar way to this:
year mean LowerCI UpperCI
r1 2005 .7031588 .69229454 .71379805
r2 2006 .75532377 .74504232 .7653212
r3 2007 .78125924 .77125096 .79094833
r4 2008 .80014324 .79059798 .80935836
r5 2009 .81860977 .80955398 .82732689
r6 2010 .82641232 .81723672 .83522016
r7 2011 .81854123 .80955547 .82719356
r8 2012 .83497983 .82621944 .8433823
r9 2013 .85411799 .84527379 .86253893
r10 2014 .84461939 .83499599 .85377985
I have tried the following commands, which give different estimates to the binomial Wilson option:
svyset id2
bysort year: eststo: ci dscomplete_binary, binomial wilson level(95)
I think the postfile family of commands will help you here. This won't save your data into a matrix, but will save the results of the ci command into a new data set, which you name and whose structure you set. After the analysis is complete, you can load the data saved by postfile and export to Excel in the manner of your choosing.
For postfile, you analyze the data in a loop instead of using by or bysort.
Assuming the years in your data run 2005-2014, here is sample code:
/*make sure no postfile is open, in case a previous run did not close the file*/
cap postclose ci_results
/*create the postfile that will store results*/
postfile ci_results year mean lowerCI upperCI using ci_results.dta, replace
/*loop through years*/
forval y = 2004/2014 {
ci dscomplete_binary if year==`y', binomial wilson level(95)
/*store saved results from ci to postfile. Make sure the post statement contains results in the same order stated in postfile command.*/
post (`y') (r(mean)) (r(lb)) (r(ub))
}
/*close the postfile once you've looped through all the cases of interest*/
postclose ci_results
use ci_results.dta, clear
Once you load the ci_results.dta data into memory, you can apply any Excel exporting command you like.
This is a development of the suggestion already made to use statsby. The objections to it are quite puzzling, as it is easy to get back to the original dataset. There is some machine time in re-loading a dataset, but how much personal time has been spent in pursuit of an alternative?
Absent a dataset which we can use, I've provided a reproducible example.
If you wish to do this repeatedly, you'll write a more elaborate program to do it, which is what this forum is all about.
I leave how to export results to Excel as a matter for those so inclined: no details of what is wanted are provided in any case.
. sysuse auto, clear
(1978 Automobile Data)
. preserve
. statsby mean=r(mean) ub=r(ub) lb=r(lb), by(rep78) : ci foreign, binomial wilson level(95)
(running ci on estimation sample)
command: ci foreign, binomial wilson
mean: r(mean)
ub: r(ub)
lb: r(lb)
by: rep78
Statsby groups
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5
.....
. list
+----------------------------------------+
| rep78 mean ub lb |
|----------------------------------------|
1. | 1 0 .6576198 0 |
2. | 2 0 .3244076 0 |
3. | 3 .1 .2562108 .0345999 |
4. | 4 .5 .7096898 .2903102 |
5. | 5 .8181818 .9486323 .5230194 |
+----------------------------------------+
. restore
. describe
The describe results will show that we are back where we started.
Related
I have tried many different combinations of sktest and sadly nothing works.
I was almost certain that sktest will work with by combination but it doesn't.
The issue is: I have binary data gender (male 0 and female 1) and I want to measure the skewness of returns for each (male and female) in the variable returns. Can you please advise?
I was hoping for a result similar to what we get when we run e.g. by gender: summarize returns
Different questions are bundled together here.
Testing
If you want to run sktest for different groups, you can just repeat the command
sysuse auto, clear
sktest price if foreign == 1
sktest price if foreign == 0
or write your own wrapper program to do the same. sktest in essence shows P-values but no summary measures.
Or do something like this:
preserve
statsby , by(foreign) : sktest price
list
restore
Measuring
If you want to see (moment-based) skewmess measures, you can just repeat summarize
bysort foreign: summarize price, detail
A wrapper is already available on SSC that is more selective.
moments price, by(foreign)
----------------------------------------------------------------------
Group | n mean SD skewness kurtosis
----------+-----------------------------------------------------------
Domestic | 52 6072.423 3097.104 1.778 5.090
Foreign | 22 6384.682 2621.915 1.215 3.555
----------------------------------------------------------------------
.
Warnings
Stata uses one estimator for moment-based skewness. There are others.
There are many ways to measure skewness. Those others mentioned in Section 7 of this paper are not a complete list; perhaps the most important omission is L-skewness (see lmoments from SSC).
I have a set of data with a dependent variable that is a count, and several independent variables. My primary independent variable is large dollar values. If I divide the dollar values by 10,000(to keep the coefficients manageable), the models(negative binomial and zero-inflated negative binomial) run in Stata and I can generate predicted counts with confidence intervals. However, theoretically it is more logical to take the natural log of this variable. When I do that, the models still run but now predicted counts on range between 0.22-0.77 or so. How do I fix this so the predicted counts generate correctly?
Your question does not show any code or data. It's nearly impossible to know what is going wrong without these two ingredients. Your questions reads as "I did some stuff to this other stuff with surprising results." In order to ask a good question, you should replicate your coding approach with a dataset that everyone would have access to, like rod93.
Here's my attempt at that, which shows reasonably similar predictions with nbreg from both models:
webuse rod93, clear
replace exposure = exposure/10000
nbreg deaths exposure age_mos, nolog
margins
predictnl d1 =predict(n), ci(lb1 ub1)
/* Compare the prediction for the first obs by hand */
di exp(_b[_cons]+_b[age_mos]*age_mos[1]+_b[exposure]*exposure[1])
di d1[1]
gen ln_exp = ln(exposure)
nbreg deaths ln_e age_mos, nolog
margins
predictnl d2 =predict(n), ci(lb2 ub2)
/* Compare the prediction for the first obs by hand */
di exp(_b[_cons]+_b[age_mos]*age_mos[1]+_b[ln_e]*ln(exposure[1]))
di d2[1]
sum d? lb* ub*, sep(2)
This produces very similar predictions and confidence intervals:
. sum d? lb* ub*, sep(2)
Variable | Obs Mean Std. Dev. Min Max
-------------+---------------------------------------------------------
d1 | 21 84.82903 25.44322 12.95853 104.1868
d2 | 21 85.0432 25.24095 32.87827 105.1733
-------------+---------------------------------------------------------
lb1 | 21 64.17752 23.19418 1.895858 80.72885
lb2 | 21 59.80346 22.01917 10.9009 79.71531
-------------+---------------------------------------------------------
ub1 | 21 105.4805 29.39726 24.02121 152.7676
ub2 | 21 110.2829 29.16468 51.76427 143.856
So I am running an 2SLS model by interview year and I have many interview years and different models. I want to present the first-stage results first and then after reassuring the reader that they are solid move on to the interesting results.
Example of Table A (first stage):
Year DV Coef SE F N
1 A 0.5 0.1 100 1000
2 A 0.8 0.2 10 1500
3 B -0.6 0.4 800 800
Table B with the main results would look the same just without the F-Stat.
I searched on the web about how to create those tables automatically in Stata, but despite finding many questions I didn't find an answer that worked for me. From those different posts and help-files I build something that is nearly there.
It creates the table I want for the main results with the F-Stat together by some variable (Step A in the code). However, when I move on to do the same for the first stage it only saves the last wave as I restore the estimates. I understand why Stata does it like that, but I cannot think of a way of convincing it to do what I want.
clear all
*Install user-written commands
ssc install outreg2, replace
ssc install ivreg210, replace
*load data
sysuse auto, clear
*run example model (obviously the model itself is bogus)
********************************************************
*Step A: creates the IV results by foreign plus the F-Statistic
bys foreign: ///
outreg2 using output1-IV-F, label excel stats(coef se) dec(2) adds(F-Test, e(widstat)) nocons nor2 keep(mpg) replace: ///
ivreg210 price headroom trunk (mpg=rep78 ), savefirst first
*Step B: creates the first stage results in a seperate table
bys foreign: ///
ivreg210 price headroom trunk (mpg=rep78 ), savefirst first
est restore _ivreg210_mpg
outreg2 using output1_1st-stage, replace keep(rep78)
cap erase output1-IV-F
cap erase output1_1st-stage
So ideally I would only run the model once and have the F-Stat in the first-stage table, but I can fix that manually. The biggest issue I have is how to store the estimates when using bysort. If anyone has any suggestions about that, I would greatly appreciate it.
Thanks!
ssc install estout
then you can store whatever result you want for later use, even after a bysort.
eststo clear
sysuse auto, clear
bysort foreign: eststo: reg price weight mpg
esttab, label nodepvar nonumber
This is a round-about solution. It works, but really isn't the proper solution I was/am looking for. The "trick" is to run the 1st stage as a separate model.
clear all
*Install user-written commands
ssc install outreg2, replace
ssc install ivreg210, replace
*load data
sysuse auto, clear
*run example model (obviously the model itself is bogus)
********************************************************
*Step A: creates the IV results by foreign plus the F-Statistic
bys foreign: ///
outreg2 using output1-IV-F, label excel stats(coef se) dec(2) adds(F-Test, e(widstat)) nocons nor2 keep(mpg) replace: ///
ivreg210 price headroom trunk (mpg=rep78 ), savefirst first
*Step B: creates the first stage results in a seperate table
bys foreign: ///
ivreg210 price headroom trunk (mpg=rep78 ), savefirst first
est restore _ivreg210_mpg
outreg2 using output1_1st-stage1, replace keep(rep78)
*************
/* NEW BIT */
*************
*Step C: creates the first stage results in a seperate table
bys foreign: ///
outreg2 using output1_1st_NEW, label excel stats(coef se) dec(2) nocons nor2 keep(rep78) replace: ///
reg mpg headroom trunk rep78
cap erase output1-IV-F
cap erase output1_1st-stage1
cap erase output1_1st_NEW
I am using EU-SILC database for 2008 for Greece. Firstly, I would like to use PE040 so as to create three dummies: primeduc for education on pre-primary AND primary school seceduc on lower secondary education +(upper) secondary + post-secondary non tertiary education and tereduc on 1st + 2nd tertiary stage.
Secondly, I would like to make a variable about working experience based on the idea exper=age-educ-6 where educ I would like sth about the years (generally) spent in education.
Any ideas of which commands I should use on stata???
What I've tried so far
About stata syntax:
tabulate PE040, gen(educ)
gen primeduc=educ1+educ2
gen seceduc=educ3+educ4+educ5
gen tereduc=educ6
Having defined lnwage as =log(PY010N/(PL060+PL070)) and age as =2008-PB140, I've tried to regress and it takes only into account 191 obs.
For your first question, I think you want a 0-1 indicator, equal to 1 if either of the indicated educational categories was recorded.
gen primeduc=educ1 | educ2
gen seceduc =educ3 |educ4 |educ5
The "|" stands for logical "or". For example, primeduc will be 1 if educ1 is 1 or educ2 is 1.
I need to modify the code below which I'm using on some CPS data to capture insurance coverage. I need to output a file with the percent covered by Census region (there are four). It should look something like this:
region n percent
1 xxx xx
2 xxx xx
3 xxx xx
4 xxx xx
I could live with two rows defining the percentages covered and not covered in each region if necessary, but I really only need the percentage covered.
Here's the code I'm using:
svyset [iw=hinswt], sdrweight(repwt1-repwt160) vce(sdr)
tempname memhold
postfile `memhold' region_rec n prop using Insurance, replace
levelsof region_rec, local(lf)
foreach x of local lf{
svy, subpop(if region_rec==`x' & age>=3 & age<=17): proportion hcovany
scalar forx = `x'
scalar prop = _b[hcovany]
matrix b = e(_N_subp)
matrix c = e(_N)
scalar n = el(c,1,1)
post `memhold' (forx) (n) (prop)
}
postclose `memhold'
use Insurance, clear
list
This is what it produces:
Survey: Proportion estimation Number of obs = 210648
Population size = 291166198
Subpop. no. obs = 10829
Subpop. size = 10965424.5
Replications = 160
_prop_1: hcovany = Not covered
--------------------------------------------------------------
| SDR
| Proportion Std. Err. [95% Conf. Interval]
-------------+------------------------------------------------
hcovany |
_prop_1 | .0693129 .0046163 .0602651 .0783607
Covered | .9306871 .0046163 .9216393 .9397349
--------------------------------------------------------------
[hcovany] not found
r(111);
I can't figure out how to get around the error message at the bottom and get it to save the results. I think a SE and CV would be a desirable feature as well, but I'm not sure how to handle that within the matrix framework.
EDIT: Additional output
+----------------------------------+
| region~c n prop se |
|----------------------------------|
| 1 9640 .9360977 2 |
| 2 12515 .9352329 2 |
| 3 14445 .8769684 2 |
| 4 13241 .8846368 2 |
+----------------------------------+
Try changing _b[hcovany] for _b[some-value-label]. To be clear, the following non-sensical example is similar to your code, but instead of using _b[sex], where sex is a variable, it uses _b[Male], where Male is a value label for sex. Subpopulation sizes and standard errors
are also saved.
clear all
set more off
webuse nhanes2f
svyset [pweight=finalwgt]
tempname memhold
tempfile results
postfile `memhold' region nsubpop maleprop stderr using `results', replace
levelsof region, local(lf)
foreach x of local lf{
svy, subpop(if region == `x' & inrange(age, 20, 40)): proportion sex
post `memhold' (`x') (e(N_subpop)) (_b[Male]) (_se[Male])
}
postclose `memhold'
use `results', clear
list
If we were to use _b[sex] instead of _b[Male], we would get the same r(111) error as in your original post.
For this example, lets see what the matrix e(b), containing the estimated proportions, looks like:
. matrix list e(b)
e(b)[1,2]
sex: sex:
Male Female
y1 .48821487 .51178513
Therefore, if we wanted to extract the proportions for females instead
of males, we could use _b[Female].
Yet another option is to save the estimation result in a matrix and use numerical subscripts:
<snip>
matrix b = e(b)
post `memhold' (`x') (b[1,2])
<snip>
There are other slight changes like the use of inrange and direct use of returned estimation results with post.
Also, you may want to take a look at help _variables and its link:
[U] 13.5 Accessing coefficients and standard errors.