i am new to pymc3. I am trying to implement an unpooled model with 3 random variables (b0,b1,b2 ~ Normal) instead of one (b) with shape= 3. The code is from the book Bayesian Modeling and Computation in Python. I think it will help me implement a more complex multilevel model.
customers = sales_df.loc[:, "customers"].values
sales_observed = sales_df.loc[:, "sales"].values
food_category = pd.Categorical(sales_df["Food_Category"])
with pm.Model() as model_sales_unpooled:
s = pm.HalfNormal("s", 20, shape=3)
b = pm.Normal("b", mu=10, sigma=10, shape=3)
m = pm.Deterministic("m", b[food_category.codes] *customers)
sales = pm.Normal("sales", mu=m, sigma=s[food_category.codes],
observed=sales_observed)
the question seems pretty easy, but I just did not find a solution yet.
I want to obtain mid prices for a certain timespan. Obviously I can calculate the mid price from bid and ask prices myself. But I wondered, whether it could be done via getTicks directly. I tried eventType = 'MID' and eventType = 'PX_MID', but both don't seem to be supported:
con <- Rblpapi::blpConnect()
Ticker <- 'AAPL US Equity'
Time <- as.POSIXct('2022-02-02 14:22:50 CET')
df <- Rblpapi::getTicks(security = Ticker,
eventType = 'MID',
startTime = Time - 5 * 60,
endTime = Time + 5 * 60)
Error in getTicks_Impl(con, security, eventType, startUTC, endUTC, setCondCodes = returnAs %in% :
Constant with value 'MID' does not exist.
The same code using eventType = 'BID' or eventType = 'ASK' or eventType = 'TRADE' works fine.
I was already looking for a list of all supported eventTypes, but I did not find anything in the documentation. Does such a list exist?
And is there a simple solution to this problem? Or is it possible, that Bloomberg does not support MID prices with Tick Data?
Thanks a lot!
I am currently stuck on below issue:
I have two tables that I have to work with, one contains financial information for vessels and the other contains arrival and departure time for vessels. I get my data combining multiple excel sheets from different folders:
financialTable
voyageTimeTable
I have to calculate the result for above voyage, and apportion the result over June, July and August for both estimated and updated.
Time in June : 4 hours (20/06/2020 20:00 - 23:59) + 10 days (21/06/2020 00:00 - 30/06/2020 23:59) = 10.1666
Time in July : 31 full days
Time in August: 1 day + 14 hours (02/08/2020 00:00 - 14:00) = 1.5833
Total voyage duration = 10.1666 + 31 + 1.5833 = 42.7499
The result for the "updated" financialItem would be the following:
Result June : 100*(10.1666/42.7499) = 23.7816
Result July : 100*(31/42.7499) = 72.5148
Result August : 100*(1.5833/42.7499) = 3.7036
sum = 100
and then for "estimated" it would be twice of everything above.
This is the format I ideally would like to get:
prorataResultTable
I have to do this for multiple vessels, with multiple timespans and several voyage numbers.
Eagerly awaiting responses, if any. Many thanks in advance.
Brds,
Not sure if you're still looking for an answer, but code below gives me your expected output:
let
financialTable = Table.FromRows({{"A", 1, "profit/loss", 200, 100}}, type table [vesselName = text, vesselNumber = Int64.Type, financialItem = text, estimated = number, updated = number]),
voyageTimeTable = Table.FromRows({{"A", 1, #datetime(2020, 6, 20, 20, 0, 0), #datetime(2020, 8, 2, 14, 0, 0)}}, type table [vesselName = text, vesselNumber = Int64.Type, voyageStartDatetime = datetime, voyageEndDatetime = datetime]),
joined =
let
joined = Table.NestedJoin(financialTable, {"vesselName", "vesselNumber"}, voyageTimeTable, {"vesselName", "vesselNumber"}, "$toExpand", JoinKind.LeftOuter),
expanded = Table.ExpandTableColumn(joined, "$toExpand", {"voyageStartDatetime", "voyageEndDatetime"})
in expanded,
toExpand = Table.AddColumn(joined, "$toExpand", (currentRow as record) =>
let
voyageInclusiveStart = DateTime.From(currentRow[voyageStartDatetime]),
voyageExclusiveEnd = DateTime.From(currentRow[voyageEndDatetime]),
voyageDurationInDays = Duration.TotalDays(voyageExclusiveEnd - voyageInclusiveStart),
createRecordForPeriod = (someInclusiveStart as datetime) => [
inclusiveStart = someInclusiveStart,
exclusiveEnd = List.Min({
DateTime.From(Date.EndOfMonth(DateTime.Date(someInclusiveStart)) + #duration(1, 0, 0, 0)),
voyageExclusiveEnd
}),
durationInDays = Duration.TotalDays(exclusiveEnd - inclusiveStart),
prorataDuration = durationInDays / voyageDurationInDays,
estimated = prorataDuration * currentRow[estimated],
updated = prorataDuration * currentRow[updated],
month = Date.MonthName(DateTime.Date(inclusiveStart)),
year = Date.Year(inclusiveStart)
],
monthlyRecords = List.Generate(
() => createRecordForPeriod(voyageInclusiveStart),
each [inclusiveStart] < voyageExclusiveEnd,
each createRecordForPeriod([exclusiveEnd])
),
toTable = Table.FromRecords(monthlyRecords)
in toTable
),
expanded =
let
dropped = Table.RemoveColumns(toExpand, {"estimated", "updated", "voyageStartDatetime", "voyageEndDatetime"}),
expanded = Table.ExpandTableColumn(dropped, "$toExpand", {"month", "year", "estimated", "updated"})
in expanded
in
expanded
The code tries to:
join financialTable and voyageTimeTable, so that for each vesselName and vesselNumber combination, we know: estimated, updated, voyageStartDatetime and voyageEndDatetime.
generate a list of months for the period between voyageStartDatetime and voyageEndDatetime (which get expanded into new table rows)
for each month (in the list), do all the arithmetic you mention in your question
get rid of some columns (like the old estimated and updated columns)
I recommend testing it with different vesselNames and vesselNumbers from your dataset, just to see if the output is always correct (I think it should be).
You should be able to manually inspect the cells in the $toExpand column (of the toExpand step/expression) to see the nested rows before they get expanded.
I created dummies in SAS (part of the codes below) and run regression (threw away M23). It was working fine. But then I tried to group them by age since we don't have enough members. I ran it the same way and threw away one age group (M20to24 since this group has the highest membership). Now some of my variables have 0 DF. Does anyone know what went wrong?
I got the message - Note: Model is not full rank. Least-squares solutions for the parameters are not unique. Some statistics will be misleading. A reported DF of 0 or B means that the estimate is biased. The following parameters have been set to 0, since the variables are a linear combination of other variables as shown.
data Table;
set Table;
M0=(AgeGender = '0M');
M1=(AgeGender = '1M');
M2=(AgeGender = '2M');
M3=(AgeGender = '3M');
M4=(AgeGender = '4M');
M5to9=(AgeGender = ' 5to9M');
M10to14=(AgeGender = '10to14M');
M15to19=(AgeGender = '15to19M');
M20to24=(AgeGender = '20to24M');
M25to29=(AgeGender = '25to29M');
M30to34=(AgeGender = '30to34M');
M35to39=(AgeGender = '35to39M');
M40to44=(AgeGender = '40to44M');
M45to49=(AgeGender = '45to49M');
M50to54=(AgeGender = '50to54M');
M55to59=(AgeGender = '55to59M');
M60to64=(AgeGender = '60to64M');
M65Plus=(AgeGender = '65+M');
F0=(AgeGender = '0F');
F1=(AgeGender = '1F');
F2=(AgeGender = '2F');
F3=(AgeGender = '3F');
F4=(AgeGender = '4F');
F5to9=(AgeGender = ' 5to9F');
F10to14=(AgeGender = '10to14F');
F15to19=(AgeGender = '15to19F');
F20to24=(AgeGender = '20to24F');
F25to29=(AgeGender = '25to29F');
F30to34=(AgeGender = '30to34F');
F35to39=(AgeGender = '35to39F');
F40to44=(AgeGender = '40to44F');
F45to49=(AgeGender = '45to49F');
F50to54=(AgeGender = '50to54F');
F55to59=(AgeGender = '55to59F');
F60to64=(AgeGender = '60to64F');
F65Plus=(AgeGender = '65+F');
Dep = (Relationship = 'Dep');
Mandatory = (Mand_Vo = 'Mandatory');
run;
ods output ParameterEstimates=Parameter_Estimates;
proc reg data= Table;
model logPMPM =
M0
M1
M2
M3
M4
M5to9
M10to14
M15to19
M25to29
M30to34
M35to39
M40to44
M45to49
M50to54
M55to59
M60to64
M65Plus
F0
F1
F2
F3
F4
F5to9
F10to14
F15to19
F20to24
F25to29
F30to34
F35to39
F40to44
F45to49
F50to54
F55to59
F60to64
F65Plus;
weight Membership;
run;
ods output close;
It doesn't look like you have overlaps or identical complimentary data variables but that's by definition. Your data is likely having that occur by chance, which is harder to find. You can likely find this by crossing variables that you suspect may be related or doing a pair wise scatter plot (PROC SGSCATTER) and seeing which two overlap almost identically.
You're correct, you wouldn't get this behaviour with continuous values because they're continuous and less likely to overlap exactly. In general, it's considered best practice to NOT categorize/bin variables when you can keep them continuous. The boundaries are artificial, does a 34 year old really differ from that 36 year old? What if all the people in that age group are 34 compared to the 36 in the 35 to 39 age group? You may not find a difference, but if your distribution was everyone at 39 vs everyone at 31 you may find more of a difference. Keeping the data continuous avoids these manufactured issues.
While trying to calculate Year over Year variance (unsuccessfully for 2 days now), I get the following error message.
EARLIER/EARLIEST refers to an earlier row context which doesn't exist.
YOY Variance = var PreviousYearPrinBal = CALCULATE(SUM(Deals[Principal Balance]),FILTER(ALL(Deals[Close Date].[Year]),Deals[Close Date].[Year] = EARLIER(Deals[Close Date].[Year])))
return
if(PreviousYearPrinBal = BLANK(), BLANK(), Deals[PrincipalBalance] - PreviousYearPrinBal)
In a different SO question, there is a different approach which gives me the following error:
A column specified in the call to function 'SAMEPERIODLASTYEAR' is not of type DATE. This is not supported.
yoy = CALCULATE([PrincipalBalance], SAMEPERIODLASTYEAR(Deals[Close Date].[Year]))
While I have some idea of what these mean, I do not have an idea of how to fix them. Here is my table.
And Here is what I expect as the result.
I've tried posting this question in Power BI community but haven't received an answer yet. Calculate Year over Year Variance.
ACTUAL DATA SAMPLE:
1) Created Year and Year Difference Column (Calculated Column)
Year = YEAR(Table1[Date])
Year Difference = Table1[Year] - Min(Table1[Year])
2) Created the Variance (Measure)
Variance =
Var current_YearDifference = SELECTEDVALUE(Table1[Year Difference])
Var Current_PrincipalBalance = CALCULATE(SUM(Table1[Principal Balance]),FILTER(ALL(Table1), Table1[Year Difference] = (current_YearDifference)))
Var Previous_PrincipalBalance = CALCULATE(SUM(Table1[Principal Balance]),FILTER(ALL(Table1), Table1[Year Difference] = (current_YearDifference - 1)))
Return if(current_YearDifference <> 0, (Current_PrincipalBalance - Previous_PrincipalBalance), 0)
3) Finally Created the Variance in terms of Percentage (Measure),
Variance in terms of Percentage =
Var current_YearDifference = SELECTEDVALUE(Table1[Year Difference])
Var Current_PrincipalBalance = CALCULATE(SUM(Table1[Principal Balance]),FILTER(ALL(Table1), Table1[Year Difference] = (current_YearDifference)))
Var Previous_PrincipalBalance = CALCULATE(SUM(Table1[Principal Balance]),FILTER(ALL(Table1), Table1[Year Difference] = (current_YearDifference - 1)))
Return if(current_YearDifference <> 0, ((Current_PrincipalBalance - Previous_PrincipalBalance) / Previous_PrincipalBalance), 0)
My Final Output
The Principal Balance has the function SUM selected on the Values Pane of the Output Table, where as the Year is Don't Summarize.
My Best Practice
Always Use Vars when creating Complex Measures to simplify the
formula.
Then return only a part of the Measure to check if the output is as expected.
Kindly let me know, if it helps or not.