I have daily data and want to convert them to weekly, using the following definition. Every Monday denotes the beginning of week i, and Sunday denotes the end of week i.
My date variable is called day and is already has %td format. I have a feeling that I should use the dow() function, combined with egen, group() but I struggle to get it quite right.
If your data are once a week and you have data for Mondays only, then your date variable is fine and all you need to do is declare delta(7) if you use tsset or xtset.
If your data are for two or more days a week and you wish to collapse or contract to weekly data, then you can convert to a suitable time basis like this:
* Example generated by -dataex-. To install: ssc install dataex
clear
input float date
22067
22068
22069
22070
22071
22072
22073
22074
22075
22076
22077
22078
22079
22080
end
format %td date
gen wdate = cond(dow(date) == 1, date, cond(dow(date) == 0, date - 6, date - dow(date) + 1))
format wdate %td
gen dow = dow(date)
list, sepby(wdate)
+-----------------------------+
| date dow wdate |
|-----------------------------|
1. | 01jun2020 1 01jun2020 |
2. | 02jun2020 2 01jun2020 |
3. | 03jun2020 3 01jun2020 |
4. | 04jun2020 4 01jun2020 |
5. | 05jun2020 5 01jun2020 |
6. | 06jun2020 6 01jun2020 |
7. | 07jun2020 0 01jun2020 |
|-----------------------------|
8. | 08jun2020 1 08jun2020 |
9. | 09jun2020 2 08jun2020 |
10. | 10jun2020 3 08jun2020 |
11. | 11jun2020 4 08jun2020 |
12. | 12jun2020 5 08jun2020 |
13. | 13jun2020 6 08jun2020 |
14. | 14jun2020 0 08jun2020 |
+-----------------------------+
In short, index weeks by the Mondays that start them. Now collapse or contract your dataset. Naturally if you have panel or longitudinal data some identifier may be involved too. delta(7) remains essential for anything depending on tsset or xtset.
There is no harm in using egen to map to successive integers, but no advantage in that either.
A theme underlying this is that Stata's own weeks are idiosyncratic, always starting week 1 on 1 January and always having 8 or 9 days in week 52. For more on weeks in Stata, see the papers here and here, which include the advice given in this answer, and much more.
Related
I am working with a dataset that includes 118,979 observations over 9 wide variables in Stata 16.0. The most prominent variable is whether a company-observation over multiple dates reports either "GPS" or "EPS". These companies can report both a "GPS" observation in a datapoint, as well as an "EPS" observation in the following datapoint. Please refer to the data overview below for further visualisation.
Datasample:
clear
input str8 cusip8 str16 cname str4 measure double actual long anndats_act float(fyear tanalyst meanforcast UE)
"87482X10" "TALMER BANCORP" "EPS" 1.21 20118 2014 29 .8686207 .3930131
"87482X10" "TALMER BANCORP" "GPS" 1.02 20479 2015 34 .8576471 .1893004
I need to drop the GPS observations (over multiple dates) once an identifier (being cusip8 in the table above) has reported an EPS over multiple dates. That is, if a company has reported GPS as well as EPS in e.g. January 1st, 2010, I want to drop the GPS observation such that the EPS is kept.
If a company only reports a GPS, and does not report an EPS during a given date, I want to keep the GPS observation in my dataset.
The following works for me (adjust your variable names as required):
. clear
. input str10(company_id measure) month day year
company_id measure month day year
1. "Company A" "EPS" 1 1 2010
2. "Company A" "GPS" 1 1 2010
3. "Company A" "GPS" 1 1 2010
4. "Company A" "GPS" 1 2 2010
5. "Company B" "EPS" 1 2 2010
6. "Company B" "GPS" 1 1 2010
7. "Company C" "GPS" 1 4 2010
8. "Company C" "EPS" 1 4 2010
9. end
.
. gen date = mdy(month,day,year)
. format date %d
. drop month day year
.
. sort company_id date measure
.
. gen both = 0
. by company_id date: replace both = 1 if measure[1] == "EPS" & measure[2] == "GPS"
(5 real changes made)
.
. list, sepby(company_id)
+----------------------------------------+
| company~d measure date both |
|----------------------------------------|
1. | Company A EPS 01jan2010 1 |
2. | Company A GPS 01jan2010 1 |
3. | Company A GPS 01jan2010 1 |
4. | Company A GPS 02jan2010 0 |
|----------------------------------------|
5. | Company B GPS 01jan2010 0 |
6. | Company B EPS 02jan2010 0 |
|----------------------------------------|
7. | Company C EPS 04jan2010 1 |
8. | Company C GPS 04jan2010 1 |
+----------------------------------------+
.
. drop if measure == "GPS" & both == 1
(3 observations deleted)
.
. list, sepby(company_id)
+----------------------------------------+
| company~d measure date both |
|----------------------------------------|
1. | Company A EPS 01jan2010 1 |
2. | Company A GPS 02jan2010 0 |
|----------------------------------------|
3. | Company B GPS 01jan2010 0 |
4. | Company B EPS 02jan2010 0 |
|----------------------------------------|
5. | Company C EPS 04jan2010 1 |
+----------------------------------------+
I've got the following table in Power BI:
Date | PersonID | Hours | Age
------------------------------|------
02-jan-18 | 4 | 8 | 3
06-jan-18 | 4 | 6 | 3
01-feb-18 | 4 | 6 | 3
05-feb-18 | 4 | 4 | 4
01-jan-18 | 5 | 6 | 3
01-feb-18 | 5 | 6 | 3
I have rows of data up until a few years back for multiple PersonID's. Most people have multiple rows per month because the data is split out on separate days. For every date, I have that person's age at the time (in this case, PersonID "4" had a birthday between feb 1st and feb 5th).
What I want to do is calculate the amount of hours PER MONTH, PER AGE. My end result should look something like this (average hours per month shown per age):
Age | Average hours per month
----------------------------------
1 | 35
2 | 31
3 | 28
4 | 28
I have no idea how to get started. How can I calculate a sum over 2 columns?
First, create a column on your table that will allow you to group by month:
MonthYear = EOMONTH(HoursAge[Date], 0)
Now you can write a measure that takes an average over a summarized table:
AvgHoursPerMonth = AVERAGEX(
SUMMARIZE(HoursAge,
HoursAge[MonthYear],
HoursAge[Age],
"MonthHours", SUM(HoursAge[Hours])),
[MonthHours])
Here's what the summarized table looks like for your given example:
This would give the following result when you put the measure into a table with age on the rows:
Age | AvgHoursPerMonth
----|-----------------
3 | 16
4 | 4
Based on the image, I would like to loop through the columns to find where there is a text mo. It updates mo with the results not the text mo. The challenge has been how to select the result in the next column different from where mo is.
Your answer to my comment above suggests to me that the question you ask reflects the wrong approach to the larger problem. Your description suggests that you have observations with a varying number of testname/testvalue pairs, such as
+----------------------------------------+
| id day test1 val1 test2 val2 |
|----------------------------------------|
| A 1 mo 11 . |
| A 2 mo 12 df 98.2 |
|----------------------------------------|
| B 1 df 98.3 mo 23 |
| B 2 mo 14 . |
+----------------------------------------+
and your objective is to produce observations that look like this
+----------------------+
| id day df mo |
|----------------------|
| A 1 . 11 |
| A 2 98.2 12 |
|----------------------|
| B 1 98.3 23 |
| B 2 . 14 |
+----------------------+
If that is the case, here is a reproducible example that you can copy, paste into Stata's Do-file Editor window, execute it, and examine the output to see how the technique avoids all the complexity you introduce by trying to use loops to accomplish the task. The reshape command is one of Stata's most powerful data management tools and it will benefit you to learn how to use it.
clear
input str8 id int day str8 test1 float val1 str8 test2 float val2
A 1 "mo" 11 "" .
A 2 "mo" 12 "df" 98.2
B 1 "df" 98.3 "mo" 23
B 2 "mo" 14 "" .
end
list, sepby(id) noobs
reshape long test val, i(id day) j(num)
drop if missing(test)
drop num
list, sepby(id) noobs
reshape wide val, i(id day) j(test) str
rename val* *
list, sepby(id) noobs
How can I delete duplicates which occur in column x but not in column y?
My dataset is as follows:
+-------+---+---+
| year | x | y |
+-------+---+---+
| 2001 | 1 | 2 |
| 2001 | 2 | 3 |
| 2001 | 2 | 3 |
| 2001 | 4 | 6 |
| 2001 | 5 | 9 |
| 2001 | 4 | 2 |
| 2001 | 4 | 9 |
+-------+---+---+
What I want is to remove the entries which occur in column y from the ones in column x.
My result would be: 1,4,5
I am currently learning Stata and I would love to know a good source for all possible commands, if this exists? So I can learn better on my own. Currently I have trouble to find good sources.
In Stata what you call columns are always called variables.
See http://www.statalist.org/forums/help#stata for general advice on how to present data examples in Stata questions. (The comments on CODE delimiters don't apply here.)
This may help. I didn't understand the role of year in your problem.
clear
input year x y
2001 1 2
2001 2 3
2001 2 3
2001 4 6
2001 5 9
2001 4 2
2001 4 9
end
rename x Datax
rename y Datay
gen long obs = _n
reshape long Data, i(obs) j(which) string
bysort Data (which) : drop if which[_N] == "y"
list
+---------------------------+
| obs which year Data |
|---------------------------|
1. | 1 x 2001 1 |
2. | 4 x 2001 4 |
3. | 7 x 2001 4 |
4. | 6 x 2001 4 |
5. | 5 x 2001 5 |
+---------------------------+
All possible commands aren't documented in a single place. Someone could write new commands all the time and they would not be documented anywhere except their help files. Did you mean that? Nor are all existing commands documented in one place: many are user-written and most of those are just documented by their help files.
Most of the official commands in Stata as supplied by StataCorp are documented in the manuals. Literally, there are also undocumented commands (I am not inventing this: see help undocumented) and there are also nondocumented commands that exist, known about because StataCorp mention them in talks or emails. To be as positive as possible: start with the manuals, bundled with your copy of Stata as .pdf files.
I have a dataset that has a date variable with missing dates.
var1
15sep2014
15sep2014
17sep2014
18sep2014
22sep2014
22sep2014
22sep2014
29sep2014
06oct2014
I aggregated the data using this command.
gen week = week(var1)
and the results look like this
var 1 week
15sep2014 37
15sep2014 37
17sep2014 38
18sep2014 38
22sep2014 38
I was wondering whether it would be possible to get the month name and year in the week variable.
In general, week() is part of the solution if and only if you define your weeks according to Stata's rules for weeks. They are
Week 1 of the year starts on January 1, regardless.
Week 2 of the year starts on January 8, regardless.
And so on, except that week 52 of the year includes 8 or 9 days, depending on
whether the year is leap or not.
Do you use these rules? I guess not. Then the simplest practice is to define a week by whichever day starts the week. If your weeks start on Sundays, then use the rule (dailydate - dow(dailydate)). If your weeks start on Mondays, ..., Saturdays, adjust the definition.
. clear
. input str9 svar1
svar1
1. "15sep2014"
2. "15sep2014"
3. "17sep2014"
4. "18sep2014"
5. "22sep2014"
6. "22sep2014"
7. "22sep2014"
8. "29sep2014"
9. "06oct2014"
10. end
. gen var1 = daily(svar1, "DMY")
. gen week = var1 - dow(var1)
. format week var1 %td
. list
+-----------------------------------+
| svar1 var1 week |
|-----------------------------------|
1. | 15sep2014 15sep2014 14sep2014 |
2. | 15sep2014 15sep2014 14sep2014 |
3. | 17sep2014 17sep2014 14sep2014 |
4. | 18sep2014 18sep2014 14sep2014 |
5. | 22sep2014 22sep2014 21sep2014 |
|-----------------------------------|
6. | 22sep2014 22sep2014 21sep2014 |
7. | 22sep2014 22sep2014 21sep2014 |
8. | 29sep2014 29sep2014 28sep2014 |
9. | 06oct2014 06oct2014 05oct2014 |
+-----------------------------------+
Much more discussion here, here and here, although the first should be sufficient.
Instead of using the week() function, I would probably use the wofd() function to transform your %td daily date into a %tw weekly date. Then you can just play with the datetime display formats to decide exactly how to format the date. For example:
gen date_weekly = wofd(var1)
format date_weekly %twww:_Mon_ccYY
That code should give you this:
var1 date_weekly
15sep2014 37: Sep 2014
15sep2014 37: Sep 2014
17sep2014 38: Sep 2014
18sep2014 38: Sep 2014
22sep2014 38: Sep 2014
This help file will be useful:
help datetime display formats
And if you want to brush up on the difference between %tw and %td dates, you might refresh yourself here:
help datetime