Athena SQL join Query not working as written - amazon-web-services

I am creating a query from a table that has different time periods. I am trying to connect them based on when the charge was created. The table has multiple IDs since each ID can have multiple periods.
Below you may find the closest join that I was able to do but sadly it isn't working as I did, since there are sometimes it checks the first ID and if the first condition which is cast(ProdPostgres.payment.charge.created_at as date) >= cast(subscription_periods_v2.period_from as date) and cast(ProdPostgres.payment.charge.created_at as date) <= cast(subscription_periods_v2.period_to as date)) is false, it is jumping to another line to check the other conditions, I do not want that, I want it keep checking all IDs for the first condition then jump to the second, then third... may someone help me out? I'm out of luck...
left join measures.subscription_periods_v2
on case when ProdPostgres.payment.charge.type = 'RENTAL_FEE'
then ProdPostgres.payment.charge.id = measures.subscription_periods_v2.charge_id
else
ProdPostgres.payment.booking.external_booking_id = measures.subscription_periods_v2.bookingid and ( (cast(ProdPostgres.payment.charge.created_at as date) >= cast(subscription_periods_v2.period_from as date) and cast(ProdPostgres.payment.charge.created_at as date) <= cast(subscription_periods_v2.period_to as date))
or cast(ProdPostgres.payment.charge.created_at as date) >= cast(subscription_periods_v2.period_to as date)
or cast(ProdPostgres.payment.charge.created_at as date) <= cast(subscription_periods_v2.period_from as date) )
I tried different conditions but none of them worked, this was the closet to getting the period that i want. and I explained everything in the above.
This image is the result that i get
This is the table where i am joining
The result should be from the first picture the first line should have in column AX 2023-01-25, and AY 2023-02-24 and AU the the created_at that i am doing the conditions with. But luckily i am getting the wrong answer :( the subscription_periods_v2 table is ordered by period from desc .

Related

Is there a way to evaluate for first day of the month and last day of the month?

I wanted to know if there is a way to evaluate dates in a manner that evaluate the 1st of the month and the last of a month
I have two columns
Start Date End Date Result
1/1/2020 1/31/2020 Standard
2/5/2020 2/15/2020 Irregular
Goal intended is to use conditional formatting to highlight dates that do not start on the 1st of a month or end on the last of a month
Tried this solution that was suggested out:
IsStandard = [End Date] = Date.EndOfMonth(Date.FromText([End Date])) and [Start Date] = Date.AddDays(Date.AddMonths(Date.EndOfMonth(Date.FromText([Start Date])),-1),1)
Expression evaluates but returns all results as "False" (NOTE: the dates are as text values so I had to add the additional Date.FromText() )
Is there a way to get the expression to evaluate for the date range from 1st month to last month as true and anything else False??
PowerQuery version: You could create a custom column as follows:
IsStandard = [End Date] = Date.EndOfMonth([End Date]) and [Start Date] = Date.AddDays(Date.AddMonths(Date.EndOfMonth([Start Date]),-1),1)
This version returns TRUE if the start date is the start of a month and the end date is the end of a month. To suit your use-case, you could invert the logic or break it up into a column to flag is/is not start of month and another to flag is/is not end of month.
The reason for the weird logic around start of month is PowerQuery provides the Date.EndOfMonth function but not a Date.StartOfMonth, so you have to take the end of a month, subtract a month, then add a day to get to the start of month.
Hopefully this helps :) .
Edit RE: always false...
This is another quirk of the language and not intuitive, but the key is that the type conversion via Date.FromText() returns a nullable date which is different from a strict Date datatype (this can be confirmed by adding a column based on the formula Date.FromText([End Date]) and examining the data type implicitly assigned to the new column, which is not Date.
Probably the simplest option would be to Change Type on the [Start Date] and [End Date] columns to Date prior to evaluating for 'standard' vs 'irregular' and remove the Date.FromText calls from the new column. This might make the fields more consistent anyway but you may run into an issue if any of the raw values are actually null or empty.
If any values are null or empty, you do a check for null to short-circuit the date conversion & check for start/end of month. If the null check fails, you would have the opportunity to then flag the record as something different from 'Standard' and 'Irregular', if that is important for your use-case.

Dax Finding date based on Criteria calculated Column

I couldn't find an answer for my issue elsewhere, so trying my luck here.
I have sales table and my final result should determine if there were sales made for same person in specific period of time, for example within 7 business days. for example:
For ID 123 I have to flag it that sale for products A,B,C where within specified period.
For ID 1234 only sales of products A and B meet the criteria product C was sold in irrelevant time frame.
I've created a date table with indicators that determine for each date if the date is a working day, but i am struggling to calculate the relevant last working day
For example: I need that for 01/01/2019 i will get 10/01/2019 date, based on NUMOFDAYS and FinalWorkday = TRUE, which basically means that i have to count NUMOFDAYS times TRUE statement for each date and return corresponding date.
After that step done I think that it would be pretty easy for me to determine if the sale for a person was made in specific time frame.
If someone can give me direction for that much appreciated
Thank you in advance
You could use a DateTable like this one:
I used the following DAX-expressions for the calculated columns:
nrDays = 7
isWorkDay = WEEKDAY('DateTable'[Date],2) < 6
rankWorkingDays = RANKX ( FILTER ( DateTable, DateTable[isWorkDay] = TRUE () ),
DateTable[Date] , , ASC )
LastWorkDay = LOOKUPVALUE ( DateTable[Date],
DateTable[isWorkDay], TRUE (),
DateTable[rankWorkingDays], DateTable[rankWorkingDays] + DateTable[nrDays])
This issue can be solved by the following, since three are non-working days/holidays we can filter them out via POWERQUERY, than add an Index Column and Another column Which is basically Index column + Number of days wanted, then simply merge duplicate of dates query on Index+number of days wanted column on Index column. And we get the correct date

Power BI cumulative count with multiple condition

Did anyone know how to convert RunningCount into Power bi Dax ?
I test on RunningTotal, Rankx but seems not working.
The [year] is just a text column not in Datetime format.
I still new on this and not sure if my explain is good enough or not. Sorry for any inconvenient cause.
I try to create measure/ calculation column in power bi from the formula below.
I need the count by Year, break by column like product, customer, rate and category.
Before I get into my solution, there are a few assumptions I had to make:
The "Year" column is supposed to be years, and not five digit numbers. So in my data I dropped the second "2" in each (i.e. "20213" -> "2013").
You may have another column in your data to break ties, but given the data you provided, there is no way to rank the first and third lines (they both have product ABC and year 2003).
Given those assumptions, here is my solution...
First, here is what my data looks like. I added an ID column so that we can see every row, even duplicates.
From there, you can simply add a new column with the following formula.
Running Count =
COUNTROWS(
FILTER(
'Data',
[ProductName] = EARLIER([ProductName]) &&
[Customer] = EARLIER([Customer]) &&
[Seller] = EARLIER([Seller]) &&
[Year] <= EARLIER([Year])
)
)
The EARLIER function is being used to specify a ProductName, Customer, etc. from the row of the table to be used in the filtering of the data. Once we have the data filtered, we can simple count the number of rows.
The final result looks like this. As noted in my second assumption, there is no way to break ties, so my numbers are slightly off from what you have in your screenshot

Split single row of data in Power BI data source into multiple rows

I have a table in a Power BI data source with a column for term start and term end date (term length can be longer than a month), along with meta data on the term. I need to report on status of purchased terms as at the end of each month. As far as I can see, the best way of accomplishing this would be to create a calculated table with an entry for each month on which a term is active at its end.
For example, an entry in the original table with the following data:
TermStartDate TermEndDate PurchaseAmount
2018-01-03 2018-04-12 100
Would end up in the calculated table as follows:
MonthPurchased PurchaseAmount
2018-01 100
2018-02 100
2018-03 100
How to accomplish this? Is there a better way than creating a separate calculated table to get this data? Any help or advise is appreciated
I managed to solve this myself, I detail the required steps below for reference:
Change start and end date column data types from Datetime to Date. <- This is needed to ensure we only generate dates on day boundaries in the next step
Add custom column with the following formula:
Month = List.Select( List.Dates([TermStartDate], Number.From([TermEndDate] - [TermStartDate]) +1, #duration(1, 0, 0, 0)), each _ = Date.EndOfMonth(_) )
This generates a list of dates between start and end, then filters to only leave the dates that are at the end of a month
Expand to new rows on the new Month column (menu at the top of the column)
Use Detect Data Type option on the Month column to change the datatype from Any to Date (for some reason I cannot manually select Date, the DataType menu option is greyed out on the Month column)

Measure in DAX to calculate YTD for chosen month only for Power BI

How to construct DAX measure to calculate sum of YTD value for specific month?
Here we have FactTable grouped by months. FactTable is filled with both Actual data and Forecast data. The only way to know when Actual end is information in table [Cut of date] in column [End of YTD]. In table [Cut of date] in column [End of YTD] – it is a single value table – we have the interesting chosen month, for which we want to see the calculation of YTD. In our case it is March. FactTable is updated irregularly every month with usually one month delay. There is no way of linking it to time functions like TODAY because of irregular update.
We would like to have a correct value of YTD displayed in yellow Card Visual for the month [End of YTD]. When we click on the slicer on "2018-03" we get almost what we want – correct value of 66 in the yellow Card. However this solution is not automatic. I want to see correct value automatically when the [End of YTD] month changes, in our case to April or then to May. I do not want it done by user.
My desperate effort can be downloaded from file: DAX YTD.pbix
I pursued the deer in various ways:
By using FILTER function in DAX measures. But it seems that the
FILTER function is to harsh. It is applied to fact table first,
selecting only one month, and then calculating YTD value wrongly. So if
there would be any option for forcing order of calculation and filtering, there would
be hope.
I tried SWITCH function to display proper result
for specific month and 0 or null for other months. Although I
succeed in this, I was not able to take advantage of it. When it
came to filtering I was as hopeless as before. BTW I would be able
to make it if SWITCH produced totals at the end of the table, but it
does not. Surprisingly.
I put some hopes in RELATED function to display proper results in the [Cut off date] table. I have not walk out of the fog so far.
I would appreciate your help.
Update before bounty.
Going to higher level. I have introduced a Category column to FactTable. Please download DAX YTD by category.pbix. So filtering gets more complex now. I would like to have correct YTD figures for Apples category.
Did you use the Date column from the Calendar table, instead of the one from FactTable?
If you use the date column from FactTable, when you apply a filter on the date, it will filter on the fact records which is in March, and then do the calculation afterwards, hence the result 33.
If you use the one from Calendar, when you apply a filter on it, it filters the records on Calendar (which you want to show in the chart), so the underlying calculation will still remain intact.
A working example:
Calendar = CALENDAR(DATE(2010, 1, 1), DATE(2020, 12, 31))
I suggest you to change the calculations of the measures to avoid missing values in some cases:
Total = SUM(FactTable[Value])
MTD = TOTALMTD([Total], 'Calendar'[Date])
YTD = TOTALYTD([Total], 'Calendar'[Date])
UPDATE:
It's much clearer to me what you want to achieve now but it still seems an XY problem to me.
I understand that you want to show the dashboard as is so that users do not need to click/input every time to see what they are supposed to see. That's why I don't get why you need to create a new table to store the Cut off date (End of YTD). How is it going to be maintained automatically?
The relative date filtering solution above actually still works in the .pbix file you've shared. If you drag the Date column from the Calendar table to visual level filters for the yellow card and add the relative date filtering, it should work as below:
For the End of YTD visual, you can use the following measure to get the first day of last calendar month, so you don't need to create another table for it:
End of YTD = EOMONTH(TODAY(), -2) + 1
And hopefully this is what you want to achieve:
Updated file for your reference.
UPDATE again:
I think you'll have to write your own YTD calculation instead of using the built-in one, so that you can make use of the cut off date you defined in another table. Here I assume that you have one and only one row in 'Cut off date'[End of YTD]. Note that I've added ALL() to the filter, so that the yellow card remains the same (66) instead of showing blank when some other rows/filters are clicked:
YTD_Special =
CALCULATE(
[Total],
FILTER(
ALL(FactTable),
FactTable[Date] >= DATE(YEAR(VALUES('Cut off date'[End of YTD])), 1, 1) &&
FactTable[Date] <= VALUES('Cut off date'[End of YTD])
)
)
I would resolve this by adding a calculated column to your Calendar table to categorise each row into either "YTD" or "Other", e.g.
Is YTD =
IF (
[Date] >= DATE ( YEAR ( DISTINCT ( 'Cut off date'[End of YTD] ) ), 1, 1 )
&& [Date] <= DISTINCT ( 'Cut off date'[End of YTD] ),
"YTD",
"Other"
)
I would then add the new Is YTD field to the Visual level filters of your Card visual, and choose YTD from the Basic filtering list. The measure shown can be your simple Total measure: SUM(FactTable[Value]).
This is a far more flexible and resuable solution than any specific measure gymnastics. You will not need an explosion of measures to apply the required logic on top of every base measure - they will all just work naturally. You can apply the filter at any level: Visual, Page, Report, or put it in a Slicer for control by the end user.
I prefer to return text results e.g. "YTD" / "Other" (rather than 1/0, True/False or Yes/No), as this allows for easy extension to other requirements e.g. "Prior YTD" (1 Jan 2017 to 1 Mar 2017). It also is clearer when used in visuals.
Actually I shouldn't claim the credit for this design - this roughly follows how Cognos Transformer's Relative Time functionality worked back in the 90s.
I did something like this in my Periodic/YTD report (last sheet): http://ciprianbusila.ro/
I have used the index value of the month selected (range 1-12) and based on this I have created a measure using max function please see the code below:
ACT = var
ACT_periodic=calculate([Value],Scenarios[Scenario]=values(Scenarios[Scenario]))
var max_month=max(Periods[Period Order])
var ACT_YTD=CALCULATE([Value],Scenarios[Scenario]=VALUES(Scenarios[Scenario]),all(Periods[Month]),Periods[Period Order]<=max_month)
var myselection=if(HASONEVALUE(MRD_view[.]),values(MRD_view[.]),"PERIODIC")
return
switch(
true(),
myselection="PERIODIC",ACT_periodic,
myselection="YEAR TO DATE",ACT_YTD,
ACT_periodic
)