I'm pretty new to Stata.
I have a set of observations of the form "Country GDP Year". I want to create a new variable GDP1960, which gives the GDP in 1960 of each country for each year:
USA $100m 1960 USA $100m 1960 $100m
USA $200m 1965 --> USA $200m 1965 $100m
Canada $60m 1960 Canada $60m 1960 $60m
What's the right syntax to make this happen? (I assume egen is involved in some mysterious way)
You've found a solution with cond(), but here's a couple of suggestions that might make modeling your data easier and help you avoid problems with issues that might arise when sorting by creating your rank variable (and I've got the egen solution that you asked about below):
Paste the code below into your do-file editor and run it:
*---------------------------------BEGIN EXAMPLE
clear
inp str20 country str10 gdp year
"USA" "$100m" 1960
"USA" "$200m" 1965
"Canada" "$60m" 1960
"Canada" "$120m" 1965
"USA" "$250m" 1970
"Mexico" "$90m" 1970
"Canada" "$800m" 1970
"Mexico" "$160m" 1960
"Mexico" "$220m" 1965
"Mexico" "$350m" 1975
end
//1. destring gdp so that we can work with it
destring gdp, ignore("$", "m") replace
//2. Create GDP for 1960 var:
bys country: g x = gdp if year==1960
bys country: egen gdp60 = max(x)
drop x
**you could also create balanced panels to see gaps in your data**
preserve
ssc install panels
panels country year
fillin country year
li //take a look at the results win. to see how filled panel data would look
restore
//3. create a gdp variable for each year (reshape the dataset)
drop gdp60
reshape wide gdp, i(country) j(year)
**much easier to use this format for modeling
su gdp1970
**here's a fake "outcome" or response variable to work with**
g outcome = 500+int((1000-500+1)*runiform())
anova outcome gdp1960-gdp1970 //or whatever makes sense for your situation
*---------------------------------END EXAMPLE
A one-line solution is
egen gdp60 = mean(gdp / (year == 1960)), by(country)
The trick here is the division by the expression year == 1960. This is true for 1960, in which case we divide by 1, which leaves the gdp for that year unchanged. It is false for all other years, in which case we divide by 0. That sounds crazy, but the consequence whenever we divide by zero is just missing values, which will be ignored by egen's mean() function.
You could use other egen functions, as in this case there should be at most one value for 1960 for each country, so e.g. max(), min(), total() should all work too. (If a country has no value for 1960, or a missing value, we will end up with missing, which is precisely as it should be.)
Discussion at http://www.stata-journal.com/article.html?article=dm0055
Well, I found a solution in the end. It relies on the fact that generate and replace work on the data in its sorted order, and that you can refer to the current observation with _n.
gen rank = 100
replace rank = 50 if year == 1960
gen gdp60 = .
sort country rank
replace gdp60 = cond(iso == iso[_n-1], gdp60[_n-1], gdp[_n])
drop rank
sort country year
EDIT: A more direct solution with the same flavour:
gen wanted = year == 1960
bysort country (wanted) : gen gdp60 = gdp[_N]
drop wanted
sort country year
Here wanted will be 1 for 1960 and 0 otherwise.
I can't think of anything shorter than these two lines:
gen temp = gdp if year == 1960
by country : egen gdp60 = max(temp)
If you want a variable for each year (e.g., gdp60, gdp61, gdp62,...) then you probably should use reshape
Related
I am working with a very large dataset (1 million obs.).
I have a string date that looks like this
key seq startdate (string)
AD07 1 August 2011
AD07 2 June 2011
AD07 3 February 2004
AD07 4 November 2004
AD07 5 2001
AD07 6 January 1998
AD5c23 1 January 2014
AD5c235 2 February 2014
AD5c235 3 2014
These are self-reported employment dates.
Some did not report the month at which they started.
But I would like to replace for AD07 the date “2001” to “January 2001”. Hence I cannot simply replace it because I would like to keep the original years but add the month in the string variable.
I started with:
levelsof start if start<="2016", local(levels)
which gives me all the years without the month from 1900 to 2016.
Now I would like to add "January" for the years without the month and keep original years.
How should I do that without using replace for every year? foreach loop?
You have a serious data quality problem if people are claiming to have started work in 1900 and every year since then! Even considering early employment starts and delayed retirement, that implies people older than the oldest established age.
Also, imputing "January" will impart bias as almost all job durations will be longer than they would have been. Real January starts will be correct, but no others: "June" or "July" or random months would make more obvious statistical sense.
That said, there is no loop needed here. You're asking for one line, say
replace startdate = "January " + startdate if length(trim(date)) == 4
or
replace startdate = "January " + startdate if real(startdate) < .
-- assuming a follow-up in converting to numeric dates. The logic there is that all year-only dates trim down to 4 characters, or (better) that feeding month names to real() will yield missings.
That said in turn, creating a new variable is better practice than over-writing one. Also, consider throwing away the month detail. Is it needed?
EDIT
You may have another problem if there are people with two or more jobs in the same year without month specifications. You don't want to impute all months in question as "January". You can check for such observations by
gen byte incomplete = real(startdate) < .
gen year = substr(trim(startdate), -4, 4)
bysort key year incomplete : gen byte multiplebad = incomplete & _N > 1
The data looks like
year 1970 1971 1972
num 3 1 4
But "graph twoway line year num" does not work here because they are not variable names.
What should I do?
You need to transpose your data to do almost anything interesting or useful. You don't say what your variable names are, but here is some technique.
clear
input str4 whatever x y z
year 1970 1971 1972
num 3 1 4
end
xpose, clear
drop in 1
rename (v?) (year num)
The names in your string variable won't get past the xpose but for your example it is trivial to rename what you get.
I am using a datasheet with about 87 countries for the years 1985 until 2004. One of my variables is Real GDP per capita. My intention is to create a new variable based on the previous, but with only 2 observations per country -- showing the average for 2 time periods.
So for 1985 I would want the average GDP for the time period 1985 - 1994, and for 1995 the average GDP for 1995 - 2004.
There is no data example, no specification of variable names and no code attempt here. But schematically
gen period = year < 1995
egen mean = mean(GDPpc), by(country period)
could be a start, or even a finish, depending on exactly what you want. If you want to be able to compare periods directly, then something like
egen mean1 = mean(GDPpc / (year < 1995)), by(country)
egen mean2 = mean(GDPpc / (year > 1994)), by(country)
tabdisp country period, c(mean) format(%2.0f)
tabdisp country, c(mean1 mean2) format(%2.0f)
will put variables side by side. See also the tag() function of egen.
Warning: None of this code was tested.
I have the following columns in my data
Firm - revenue - industry - year
I want to calculate the percentage change in total revenue for each industry between 2008 and 2015.
I tried:
by industry: egen tot_2008 = sum(revenue) if year == 2008
by industry: egen tot_2015 = sum(revenue) if year == 2015
gen change = (tot_2015-tot_2008)/tot_2008
But this doesn't work as the ifs restrict which years the egen creates values for as well as which years are included in each sum.
As you realise, the problem with your code is that 2008 and 2015 values will be non-missing values only for those years respectively, and hence never not missing on both variables. Here is one way to spread values to all years for each industry:
by industry: egen tot_2008 = total(revenue / (year == 2008))
by industry: egen tot_2015 = total(revenue / (year == 2015))
gen change = (tot_2015-tot_2008)/tot_2008
That hinges on expressions such as year == 2008 being evaluated as 1 if true and 0 if false. If you divide by 0, the result is a missing value, which Stata ignores, which is exactly what you want. Taking totals over all observations in an industry ensures that the same value is recorded for each industry.
Here is another way that some find more explicit:
by industry: egen tot_2008 = total(cond(year == 2008, revenue, .))
by industry: egen tot_2015 = total(cond(year == 2015, revenue, .))
gen change = (tot_2015-tot_2008)/tot_2008
which hinges on the same principle, that missings will be ignored.
Note the use of the egen function total() here. The egen function sum() still works, and is the same function, but that name is undocumented as of Stata 9, in an attempt to avoid confusion with the Stata function sum().
To avoid double (indeed multiple) counting, use
egen tag = tag(industry)
to tag just one observation for each industry, to be used in graphs and tables for which you want that.
For discussion, see here, sections 9 and 10.
I am exploring an effect that I think will vary by GDP levels, from a data set that has, vertically, country and year (1960 to 2015), so each country label is on 55 rows. I ran
sort year
by year: egen yrank = xtile(rgdp), nquantiles(4)
which tags every year row with what quartile of GDP they were in that year. I want to run this:
xtreg fiveyearg taxratio if yrank == 1 & year==1960
which would regress my variable (tax ratio) against some averaged gdp data from countries that were in the bottom quartile of GDPs in 1960 alone. So even if later on they grew enough to change ranks, the later data would still be in the regression pool. Sadly, I cannot get this code, or any variation, to run.
My current approach is to try to generate some new variable that would give every row with country label X a value of 1 if they were in the bottom quartile in 1960, but I can't get that to work either. i have run out of ideas, so I thought I would ask!
Based on your latest comment, which describes the (un)expected behavior:
clear
set more off
*----- example data -----
input ///
country year rank
1 1960 2
1 1961 1
1 1962 2
2 1960 1
2 1961 1
2 1962 1
3 1960 3
3 1961 3
3 1962 3
end
list, sepby(country)
*----- what you want -----
// tag countries whose first observation for -rank- is 1
// (I assume the first observation for -year- is always 1960)
bysort country : gen toreg = rank[1] == 1
list, sepby(country)
// run regression conditional on -toreg-
xtreg ... if toreg
Check help subscripting if in doubt.