Pivot with dynamic DATE columns - oracle-apex

I have a query that I created from a table.
example:
select
pkey,
trunc (createdformat) business_date,
regexp_substr (statistics, 'business_ \ w *') business_statistics
from business_data
where statistics like '% business_%'
group by regexp_substr(statistics, 'business_\w*'), trunc(createdformat)
This works great thanks to your help.
Now I want to show that in a crosstab / pivot.
That means in the first column are the "business_statistics", the column headings are the "dynamic days from business_date".
I've tried the following, but it doesn't quite work yet
SELECT *
FROM (
select
pkey,
trunc(createdformat) business_date,
regexp_substr(statistics, 'business_\w*') business_statistics
from business_data
where statistics like '%business_%'
)
PIVOT(
count(pkey)
FOR business_date
IN ('17.06.2020','18.06.2020')
)
ORDER BY business_statistics
If I specify the date, like here 17.06.2020 and 18.06.2020 it works. 3 columns (Business_Statistic, 17.06.2020, 18.06.2020). But from column 2 it should be dynamic. That means he should show me the days (date) that are also included in the query / table. So that is the result of X columns (Business_Statistics, Date1, Date2, Date3, Date4, ....). Dynamic based on the table data.
For example, this does not work:
...
IN (SELECT DISTINCT trunc(createdformat) FROM BUSINESS_DATA WHERE statistics like '%business_%' order by trunc(createdformat))
...

The pivot clause doesn't work with dynamic values.
But there are some workarounds discuss here: How to Convert Rows to Columns and Back Again with SQL (Aka PIVOT and UNPIVOT)
You may find one workaround that suits your requirements.

Unfortunately, I am not very familiar with PL / SQL. But could I still process the start date and the end date of the user for the query?
For example, the user enters the APEX environment as StartDate: June 17, 2020 and as EndDate: June 20, 2020.
Then the daily difference is calculated in the PL / SQL query, then a variable is filled with the value of the entered period using Loop.
Example: (Just an idea, I'm not that fit in PL / SQL yet)
DECLARE
startdate := :P9999_StartDate 'Example 17.06.2020
enddate := P9999_EndDate 'Example 20.06.2020
BEGIN
LOOP 'From the startdate to the enddate day
businessdate := businessdate .... 'Example: 17.06.2020,18.06.2020,19.06.2020, ...
END LOOP
SELECT *
FROM (
select
pkey,
trunc(createdformat) business_date,
regexp_substr(statistics, 'business_\w*') business_statistics
from business_data
where statistics like '%business_%'
)
PIVOT(
count(pkey)
FOR business_date
IN (businessdate)
)
ORDER BY business_statistics
END;
That would be my idea, but I fail to implement it. Is that possible? I hope you understand what I mean

Related

IF 'AND-OR' ISFILTERED combination in DAX giving problems

Below is the sample dataset
The data has two slicers ( date and category ) shown below
I am writing a DAX Statement to multiply the sum(values) * 10 only if the date range is in the current year 2023.
The StartYear gives the start of the current year, firstD gives the lowest date from the date slicer.
Formula =
var new = sum(Test[Value]) * 10
var startyear = DATE(YEAR(TODAY()),1,1)
var firstD = CALCULATE( MIN( Test[Date]), ALLSELECTED(Test[Date]) )
return if( ISFILTERED(Test[Categories]) && firstD >= startyear, new, 0 )
Now when I filter dates to 2023, the total value should be 2300 but it shows as 0
However the DAX works when I select A or B
If we remove the ISFILTERED function then, it gives wrong value, the expected value is 0 because the start date is in 2022, but it shows 650
let me know if that is the right syntax
It looks like you are not using a separate calendar table to handle this, which you need!
In your very last example you have set your slicer to some time late 2022, but the minimum value of 'Test'[Date] for your selected category is in year 2023. Hint: set the slicer to e.g. 2022-12-14, this will include a 2022-date for Category A in your data.
Your measure behaves exactly how it is supposed to, in other words!
To fix this, you need to do the following:
Create a calendar table in your model, this should contain contiguous dates, which is necessary for the filtering method you want
Establish a relationship between the calendar table and existing Test table.
Use the date column from your new calendar table in your slicer and as date reference in your measure
Exactly how to create a calendar table is thoroughly documented on Google, I recommend you search and find an article or video you understand for implementing this.
Lastly: Your use of ISFILTERED in this measure seems strange, since you mention nowhere the requirement of only showing a number if the column you are testing filtering on is filtered, if that makes sense.. :-) The way you describe your calculation, you only need to check whether the selected date range starts in current year.

i am looking to get the date diff from two or more row in a way that first rows serviceto date - second rows service start date so that i can get diff

my data looks like this
userid
completedat
serviceperiodfrom
serviceperiodto
00002cd9-94eb-4c06-a2c4-75253fd541b9
2020-11-25T14:20:04.293Z
2020-11-25T14:20:04.200Z
2021-02-25T14:20:04.200Z
00002cd9-94eb-4c06-a2c4-75253fd541b9
2021-03-21T10:27:34.842Z
2021-03-21T10:27:34.800Z
2022-03-21T10:27:34.800Z
00002cd9-94eb-4c06-a2c4-75253fd541b9
2020-07-24T11:22:12.410Z
2020-07-24T11:22:12.300Z
2020-10-24T11:22:12.300Z
I need the date diff from serviceperiodto date of first row - serviceperiodfrom date of secondrow and it goes for as many iteration as it has these details for each userid
please help me i tried joining the tables using subqueries tried to create a pivot table but none of them seem working for me please help
You can use lag/lead to access previous/next item:
WITH dataset
AS (SELECT *
FROM
(
VALUES
(1, from_iso8601_timestamp('2020-11-25T14:20:04.200Z'), from_iso8601_timestamp('2021-02-25T14:20:04.200Z')),
(1, from_iso8601_timestamp('2021-03-21T10:27:34.800Z'), from_iso8601_timestamp('2022-03-21T10:27:34.800Z')),
(1, from_iso8601_timestamp('2020-07-24T11:22:12.300Z'), from_iso8601_timestamp('2020-10-24T11:22:12.300Z'))
) AS t (userid, serviceperiodfrom, serviceperiodto)
)
SELECT date_diff(
'hour',
serviceperiodto,
lead(serviceperiodfrom, 1) OVER (PARTITION BY userid ORDER BY serviceperiodfrom))
FROM dataset
Output:
_col0
770
572
 

How to find missing dates in BigQuery table using sql

How to get a list of missing dates from a BigQuery table. For e.g. a table(test_table) is populated everyday by some job but on few days the jobs fails and data isn't written into the table.
Use Case:
We have a table(test_table) which is populated everyday by some job( a scheduled query or cloud function).Sometimes those job fail and data isn't available for those particular dates in my table.
How to find those dates rather than scrolling through thousands of rows.
The below query will return me a list of dates and ad_ids where data wasn't uploaded (null).
note: I have used MAX(Date) as I knew dates was missing in between my boundary dates. For safe side you can also specify the starting_date and ending_date incase data hasn't been populated in the last few days at all.
WITH Date_Range AS
-- anchor for date range
(
SELECT MIN(DATE) as starting_date,
MAX(DATE) AS ending_date
FROM `project_name.dataset_name.test_table`
),
day_series AS
-- anchor to get all the dates within the range
(
SELECT *
FROM Date_Range
,UNNEST(GENERATE_TIMESTAMP_ARRAY(starting_date, ending_date, INTERVAL 1 DAY)) AS days
-- other options depending on your date type ( mine was timestamp)
-- GENERATE_DATETIME_ARRAY or GENERATE_DATE_ARRAY
)
SELECT
day_series.days,
original_table.ad_id
FROM day_series
-- do a left join on the source table
LEFT JOIN `project_name.dataset_name.test_table` AS original_table ON (original_table.date)= day_series.days
-- I only want the records where data is not available or in other words empty/missing
WHERE original_table.ad_id IS NULL
GROUP BY 1,2
ORDER BY 1
Final output will look like below:
An Alternate solution you can try following query to get desired output:-
with t as (select 1 as id, cast ('2020-12-25' as timestamp) Days union all
select 1 as id, cast ('2020-12-26' as timestamp) Days union all
select 1 as id, cast ('2020-12-27' as timestamp) Days union all
select 1 as id, cast ('2020-12-31' as timestamp) Days union all
select 1 as id, cast ('2021-01-01' as timestamp) Days union all
select 1 as id, cast ('2021-01-04' as timestamp) Days)
SELECT *
FROM (
select TIMESTAMP_ADD(Days, INTERVAL 1 DAY) AS Days, TIMESTAMP_SUB(next_days, INTERVAL 1 DAY) AS next_days from (
select t.Days,
(case when lag(Days) over (partition by id order by Days) = Days
then NULL
when lag(Days) over (partition by id order by Days) is null
then Null
else Lead(Days) over (partition by id order by Days)
end) as next_days
from t) where next_days is not null
and Days <> TIMESTAMP_SUB(next_days, INTERVAL 1 DAY)),
UNNEST(GENERATE_TIMESTAMP_ARRAY(Days, next_days, INTERVAL 1 DAY)) AS days
Output will be as :-
I used the code above but had to restructure it for BigQuery:
-- anchor for date range - this will select dates from the source table (i.e. the table your query runs off of)
WITH day_series AS(
SELECT *
FROM (
SELECT MIN(DATE) as starting_date,
MAX(DATE) AS ending_date
FROM --enter source table here--
---OPTIONAL: filter for a specific date range
WHERE DATE BETWEEN 'YYYY-MM-DD' AND YYYY-MM-DD'
),UNNEST(GENERATE_DATE_ARRAY(starting_date, ending_date, INTERVAL 1 DAY)) as days
-- other options depending on your date type ( mine was timestamp)
-- GENERATE_DATETIME_ARRAY or GENERATE_DATE_ARRAY
)
SELECT
day_series.days,
output_table.date
FROM day_series
-- do a left join on the output table (i.e. the table you are searching the missing dates for)
LEFT JOIN `project_name.dataset_name.test_table` AS output_table
ON (output_table.date)= day_series.days
-- I only want the records where data is not available or in other words empty/missing
WHERE output_table.date IS NULL
GROUP BY 1,2
ORDER BY 1

How to convert YYYYMM to datetime or number?

I have data in the powerbi loaded from azure analytics service. As I connect it in live mode, so I cannot open the power query in the report.
Now I need to convert YYYYMM (called calendar from the table, for instance, 201905) to number or datetime. As I do not know the default format, so I try to use date or value to get the result. However, I am not able to find the table in the formula. For example:
Value(table[calendar])
or
Date = DATE ( LEFT ( table[calendar], 4 ), RIGHT ( table[calendar], 2 ), 1 )
My question is how to convert YYYYMM to the NUMBER or DATE. Thank you.
As you are creating a measure, you first need to make sure that it properly retrieves a date from the current table row.
mDate =
var ThisDate = MAX(tbl_YYYYMM[Dates])
return
ThisDate
Now that we know it works properly, you can amend your measure as follows:
mDate =
var ThisDate = MAX(tbl_YYYYMM[Dates])
return
DATE(
LEFT(ThisDate, 4),
RIGHT(ThisDate, 2),
1
)
You might now want to change the measure's datatype so that it doesn't display time:
P.S. I am wondering, is measure really your only option? As suggested in the comments, it might be easier to perform this transformation in DAX Calculated Column, or even at Power Query level.

Sorting date filter in DESC order in Power BI

I have a date filter for my dashboard and depending on which end of month date the user selects the dashboard displays the appropriate values for the selected month and all of that works correctly. However I would like my date filter to be sorted in DESC order for the user so they do not have to scroll down to the bottom to get the most recent month. I have sorted the dataset in DESC order for the query tied to my dashboard, I have gone to the DATA tab within Power BI and sorted the dataset in DESC order but no matter what I try the filter will not sort in DESC order. Any help is much appreciated.
Unfortunately in page level filters there is no option to sort date to descending order.
One option is to use a Date slicer and sort it descending.
But if you cant have date slicer then a work around is there to implement this.
For this you need to create two calculated columns. One a duplicate column of your date field and another to sort this field.
SortCol = Table[EOM Date]*-1
EOMDateCopy=LEFT([EOM Date],LEN([EOM Date]))
Sort the EOMDateCopy column with SortCol and drag EOMDateCopy in your page level filter.
Best Regards,
Shani
My suggestion is to build your own Calendar table. In "Data" View, create a new Table by writing DAX similar like this:
Calendar =
VAR _MinDate = MIN('YourFactTable'[Date])
VAR _MaxDate = MAX('YourFactTable'[Date])
VAR _BaseCalendar = CALENDAR(_MinDate,_MaxDate)
RETURN
GENERATE(
_BaseCalendar,
VAR _BaseDate = [Date]
VAR _Year = YEAR(_BaseDate)
VAR _Month = MONTH(_BaseDate)
VAR _Quarter = QUARTER(_BaseDate)
VAR _Day = DAY(_BaseDate)
RETURN
ROW (
"Calendar Year", _Year,
"Calendar Month", _Month,
"Calendar Year-Month", _Year&"-"&IF(LEN(_Month)=1,"0"&_Month,_Month),
"Calendar Quarter", FORMAT(_Quarter,"\Q0"),
"Date Order", 9999 - (_Year+_Month+_Day)
)
)
"Date Order", 9999 - (_Year+_Month+_Day) create a descending order for your date.
Next, build a relationship between your Calendar table and your Fact table by relating the "Date" column of your Calendar table and your date column (version_no in my case) of your Fact table. Make sure these two columns have exactly the same date format.
Finally, click on the column and set "Sort by column" as Date Order
You will see your date filter is sorted in desc.
Please make sure your date field is in proper format and then try to sort.
If it is already in date format, please give a bit more details so that we can check.
Best Regards,
Shani