I'm trying to create unique IDs based on some conditions, but I really do not know how to get this the way I expect.
I've got two important columns "Concepto" (Concept) and ID (which is where I wanna carry out the transformation and it's a duplicate of Concepto. The table is as follows:
Concepto
ID
TARJETA ENT: REF:08780807807 8603CDTO. BUSINESS 1040-25/12/2021
TARJETA ENT: REF:08780807807 8603CDTO. BUSINESS 1040-25/12/2021
TARJETA ENT:-REF:08780810810 8603CDTO. BUSINESS 1040-25/12/2021
TARJETA ENT:-REF:08780810810 8603CDTO. BUSINESS 1040-25/12/2021
INTERESES DE PRESTAMO PTMO.:000087816490219968
INTERESES DE PRESTAMO PTMO.:000087816490219968
AMORTIZACION PRESTAMO PTMO.:000087816490219968
AMORTIZACION PRESTAMO PTMO.:000087816490219968
INTERESES DE LEASING LEAS.:000087816520001372
INTERESES DE LEASING LEAS.:000087816520001372
AMORTIZACION LEASING LEAS.:000087816520001372
AMORTIZACION LEASING LEAS.:000087816520001372
LEASING-COBRO DEL IVA LEAS.:000087816520001372
LEASING-COBRO DEL IVA LEAS.:000087816520001372
AN.NOMINAL REMESA EFECTOS ES8230580878030689003155-6030865924
AN.NOMINAL REMESA EFECTOS ES8230580878030689003155-6030865924
IMPORTE ORDEN PAGO CONTRATO: 000087806860000063
IMPORTE ORDEN PAGO CONTRATO: 000087806860000063
The excepted results should be something like:
Concepto
ID
TARJETA ENT: REF:08780807807 8603CDTO. BUSINESS 1040-25/12/2021
0878080780725122021
TARJETA ENT:-REF:08780810810 8603CDTO. BUSINESS 1040-25/12/2021
0878081081025122021
INTERESES DE PRESTAMO PTMO.:000087816490219968
000087816490219968
AMORTIZACION PRESTAMO PTMO.:000087816490219968
000087816490219968
INTERESES DE LEASING LEAS.:000087816520001372
000087816520001372
AMORTIZACION LEASING LEAS.:000087816520001372
000087816520001372
LEASING-COBRO DEL IVA LEAS.:000087816520001372
000087816520001372
AN.NOMINAL REMESA EFECTOS ES8230580878030689003155-6030865924
6030865924
IMPORTE ORDEN PAGO CONTRATO: 000087806860000063
000087806860000063
Thanks in advance!
If you are ok with making the ID as just the numbers from Concepto:
Table.AddColumn(
...
,"ID"
, each Text.Select([Concepto],{"0".."9"})
)
https://learn.microsoft.com/en-us/powerquery-m/text-select
If not - seems like you would need to:
Text.SplitAnyon -,:,space
List.Select items that are (only numbers and >4 characters long) OR (numbers and / )
Select number text from those
... Which can all be done inside Table.AddColumn either as nested function block or an inline function declaration
The problem here, as has been pointed out, is that you are trying to extract data with varying delimiters.
Starting with what you provided, I was able to get to a final data set that I believe is what you are looking for. That said, the caveat is that one step involves removing a specific phrase ("COBRO DEL IVA LEAS."). If that phrase varies, you'll need to find a work around for it.
Here's the M code - the steps can probably be rearranged, but I'll leave that up to you:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Split Column by Delimiter" = Table.SplitColumn(Source, "ID", Splitter.SplitTextByDelimiter(":", QuoteStyle.Csv), {"ID.1", "ID.2", "ID.3"}),
#"Changed Type" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"ID.1", type text}, {"ID.2", type text}, {"ID.3", type text}}),
#"Replaced Value" = Table.ReplaceValue(#"Changed Type","REF","",Replacer.ReplaceText,{"ID.2"}),
#"Replaced Value1" = Table.ReplaceValue(#"Replaced Value","-","",Replacer.ReplaceText,{"ID.2"}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Replaced Value1", "ID.1", Splitter.SplitTextByDelimiter("-", QuoteStyle.Csv), {"ID.1.1", "ID.1.2"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter1",{{"ID.1.1", type text}, {"ID.1.2", type text}}),
#"Removed Columns" = Table.RemoveColumns(#"Changed Type1",{"ID.1.1"}),
#"Split Column by Delimiter2" = Table.SplitColumn(#"Removed Columns", "ID.3", Splitter.SplitTextByDelimiter("-", QuoteStyle.Csv), {"ID.3.1", "ID.3.2"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Split Column by Delimiter2",{{"ID.3.1", type text}, {"ID.3.2", type text}}),
#"Split Column by Delimiter3" = Table.SplitColumn(#"Changed Type2", "ID.3.1", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), {"ID.3.1.1", "ID.3.1.2", "ID.3.1.3", "ID.3.1.4"}),
#"Removed Columns1" = Table.RemoveColumns(#"Split Column by Delimiter3",{"ID.3.1.2", "ID.3.1.3", "ID.3.1.4"}),
#"Replaced Value2" = Table.ReplaceValue(#"Removed Columns1","/","",Replacer.ReplaceText,{"ID.3.2"}),
#"Merged Columns" = Table.CombineColumns(#"Replaced Value2",{"ID.3.1.1", "ID.3.2"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Merged"),
#"Replaced Value3" = Table.ReplaceValue(#"Merged Columns","COBRO DEL IVA LEAS.","",Replacer.ReplaceText,{"ID.1.2"}),
#"Merged Columns1" = Table.CombineColumns(#"Replaced Value3",{"ID.1.2", "ID.2", "Merged"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Merged.1"),
#"Renamed Columns" = Table.RenameColumns(#"Merged Columns1",{{"Merged.1", "ID"}})
in
#"Renamed Columns"
It produces this table:
+-----------------------------------------------------------------+----------------------+
| Concepto | ID |
+-----------------------------------------------------------------+----------------------+
| TARJETA ENT: REF:08780807807 8603CDTO. BUSINESS 1040-25/12/2021 | 0878080780725122021 |
| TARJETA ENT:-REF:08780810810 8603CDTO. BUSINESS 1040-25/12/2021 | 0878081081025122021 |
| INTERESES DE PRESTAMO PTMO.:000087816490219968 | 000087816490219968 |
| AMORTIZACION PRESTAMO PTMO.:000087816490219968 | 000087816490219968 |
| INTERESES DE LEASING LEAS.:000087816520001372 | 000087816520001372 |
| AMORTIZACION LEASING LEAS.:000087816520001372 | 000087816520001372 |
| LEASING-COBRO DEL IVA LEAS.:000087816520001372 | 000087816520001372 |
| AN.NOMINAL REMESA EFECTOS ES8230580878030689003155-6030865924 | 6030865924 |
| IMPORTE ORDEN PAGO CONTRATO: 000087806860000063 | 000087806860000063 |
+-----------------------------------------------------------------+----------------------+
Hope that resolves it for you.
Related
I'm just starting to learn Power BI and I'm facing an issue which I think is easy to solve.
I have 3 tables linked by the column CCA3. The third table contains a column for some years showing the number of poeple for each country at this date.
I want to make a dashboard where I may show the evolution of the population of the different countries over the time, without loosing the link with the other tables. So I cannot transpose the table to get the years in a column because in this case the column CCA3 disapear and the link is broken.
Original dataset is available on the following link : https://www.kaggle.com/datasets/iamsouravbanerjee/world-population-dataset/download?datasetVersionNumber=3
The M code generated by Power Query is the following :
let
Source = Csv.Document(File.Contents("C:\Users\benny\Desktop\world_population.csv"),[Delimiter=",", Columns=17, Encoding=65001, QuoteStyle=QuoteStyle.None]),
#"En-têtes promus" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Type modifié" = Table.TransformColumnTypes(#"En-têtes promus",{{"Rank", Int64.Type}, {"CCA3", type text}, {"Country", type text}, {"Capital", type text}, {"Continent", type text}, {"2022 Population", Int64.Type}, {"2020 Population", Int64.Type}, {"2015 Population", Int64.Type}, {"2010 Population", Int64.Type}, {"2000 Population", Int64.Type}, {"1990 Population", Int64.Type}, {"1980 Population", Int64.Type}, {"1970 Population", Int64.Type}, {"Area (km²)", Int64.Type}, {"Density (per km²)", type text}, {"Growth Rate", type text}, {"World Population Percentage", type text}}),
#"Lignes triées" = Table.Sort(#"Type modifié",{{"Rank", Order.Ascending}}),
#"Valeur remplacée" = Table.ReplaceValue(#"Lignes triées",".",",",Replacer.ReplaceText,{"Density (per km²)"}),
#"Valeur remplacée1" = Table.ReplaceValue(#"Valeur remplacée",".",",",Replacer.ReplaceText,{"Growth Rate"}),
#"Type modifié1" = Table.TransformColumnTypes(#"Valeur remplacée1",{{"Growth Rate", type number}, {"Density (per km²)", type number}}),
#"Valeur remplacée2" = Table.ReplaceValue(#"Type modifié1",".",",",Replacer.ReplaceText,{"World Population Percentage"}),
#"Type modifié2" = Table.TransformColumnTypes(#"Valeur remplacée2",{{"World Population Percentage", type number}}),
#"Soustrait de la colonne" = Table.TransformColumns(#"Type modifié2", {{"Growth Rate", each _ - 1, type number}}),
#"Autres colonnes supprimées" = Table.SelectColumns(#"Soustrait de la colonne",{"Rank", "CCA3", "2022 Population", "2020 Population", "2015 Population", "2010 Population", "2000 Population", "1990 Population", "1980 Population", "1970 Population", "Area (km²)", "Density (per km²)", "Growth Rate", "World Population Percentage"}),
#"Colonnes permutées" = Table.ReorderColumns(#"Autres colonnes supprimées",{"CCA3", "Rank", "2022 Population", "2020 Population", "2015 Population", "2010 Population", "2000 Population", "1990 Population", "1980 Population", "1970 Population", "Area (km²)", "Density (per km²)", "Growth Rate", "World Population Percentage"}),
#"Autres colonnes supprimées1" = Table.SelectColumns(#"Colonnes permutées",{"CCA3", "Area (km²)", "Density (per km²)", "Growth Rate", "World Population Percentage"}),
#"Duplication de la colonne" = Table.DuplicateColumn(#"Autres colonnes supprimées1", "World Population Percentage", "World Population Percentage - Copier"),
#"Cube calculé" = Table.TransformColumns(#"Duplication de la colonne",{{"World Population Percentage - Copier", each Number.Power(_, 3), type number}}),
#"Colonnes renommées" = Table.RenameColumns(#"Cube calculé",{{"World Population Percentage - Copier", "WPP Size"}}),
#"Duplication de la colonne1" = Table.DuplicateColumn(#"Colonnes renommées", "WPP Size", "WPP Size - Copier"),
#"Colonnes supprimées" = Table.RemoveColumns(#"Duplication de la colonne1",{"WPP Size - Copier", "WPP Size"}),
#"Type modifié3" = Table.TransformColumnTypes(#"Colonnes supprimées",{{"World Population Percentage", Percentage.Type}, {"Growth Rate", Percentage.Type}}),
#"Colonne divisée" = Table.TransformColumns(#"Type modifié3", {{"World Population Percentage", each _ / 100, Percentage.Type}})
in
#"Colonne divisée"
Is there any way to do it with Power BI ?
Thanks a lot.
Cheers :)
If the year (number) is a column name it is called a "pivoted table" (Excel Style) which brings you nowhere in Power BI.
But in Power Query you can easily transform - unpivot all the YYYY Population columns and you receive 2 new columns "Attribute" containing the years and "Value" containing the population numbers.
Now you simply have to extract the year numbers from the "Attribute" column
I am trying to calculate, working hours of forklifts on a particular day based on their ON/OFF ignition status.
My code is perfectly working fine when I ignore the condition were either ON or OFF are missing.
Requirement:
Need to handle boudary conditions were last entry of the day for a device is ON
or you can say , if last entry off a day is ON, then we need to calculate its working hrs. till 12 PM midnight.And subsequently add working hrs. in next day as well
Dataset :
DEVICEID : DATETIME: IGNITIONSTATUS
1 1/11/2020 8:00 ON
1 1/11/2020 14:00 OFF
2 10/28/2020 7:54 ON
2 10/28/2020 12:23 OFF
2 10/28/2020 14:01:44 PM ON
2 10/29/2020 20:01 ON
2 10/29/2020 14:01:44 PM OFF
1 12/10/2020 8:27 ON
1 12/10/2020 15:44:22 PM OFF
let
Source = Excel.Workbook(File.Contents("C:\Users\avyas2\Desktop\DataToTest.xlsx"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"DEVICEID", Int64.Type}, {"DATETIME", type datetime}, {"IGNITIONSTATUS", type text}}),
#"Split DateTime" = Table.SplitColumn(#"Changed Type", "DATETIME", (dt) => {DateTime.Date(dt), DateTime.Time(dt)}, {"Date", "Time"}),
#"Group by Date" = Table.Group(#"Split DateTime", {"DEVICEID", "Date"}, {{
"AvailabilityStatus",
each
let
times = _[[Time], [IGNITIONSTATUS]],
ins = List.Sort(Table.SelectRows(times, each [IGNITIONSTATUS] = "ON")[Time]),
outs = List.Sort(Table.SelectRows(times, each [IGNITIONSTATUS] = "OFF")[Time]),
zip = List.Select(List.Zip({ins, outs}), each _{0} <> null and _{1} <> null )
in
zip
, type {{time}}
}}),
#"Split AvailabilityStatus" = Table.SplitColumn(Table.ExpandListColumn(#"Group by Date", "AvailabilityStatus"), "AvailabilityStatus", (x) => x, {"ON", "OFF"}),
#"Added Worked Time" = Table.AddColumn(#"Split AvailabilityStatus", "Hours Worked", each Duration.TotalHours([OFF] - [ON]), type number),
#"Renamed Columns" = Table.RenameColumns(#"Added Worked Time",{{"Hours Worked", "WorkingHrs"}})
in
#"Renamed Columns"
Maybe You need only a DAX measure?
WorkingHour =
var _thisLine = SELECTEDVALUE(Sheet2[DATETIME] )
var result = if(SELECTEDVALUE(Sheet2[IGNITIONSTATUS] )="ON",
var minDate = CALCULATE( min(Sheet2[DATETIME]), filter(ALL(Sheet2), Sheet2[DEVICEID] = SELECTEDVALUE(Sheet2[DEVICEID] ) && Sheet2[DATETIME] > _thisLine && Sheet2[IGNITIONSTATUS] ="OFF"
&& DATE(YEAR(Sheet2[DATETIME]),MONTH(Sheet2[DATETIME]),DAY(Sheet2[DATETIME])) = DATE(YEAR(_thisLine),MONTH(_thisLine),DAY(_thisLine))
))
return
if( ISBLANK(minDate),DATE(YEAR(_thisLine),MONTH(_thisLine),DAY(_thisLine)) +1 ,minDate) - _thisLine
)
return
result
I have an ID column and I am looking for ways to increment my IDs each time a specific item appears in my Geography column (ItalyZ, ItalyM, UKY or UKM) is found.
The ID of ItalyZ starts at 0 and ends at 4000.
The ID of ItalyB starts at 4000 and ends at 8000.
The ID of UKY starts at 0 and ends at 4000.
The ID of UKM starts at 4000 and ends at 8000.
However, I am refreshing my file, and I will thus have from time to time new arrivals of "geographies" without the origins or first IDs. These boundaries/ranges are only known beginning and ends.
Here is a sample of my data:
|---------------------|------------------|
| ID | Geography |
|---------------------|------------------|
| AB0000 | ItalyZ |
|---------------------|------------------|
| AB4041 | ItalyB |
|---------------------|------------------|
| BC0000 | UKY |
|---------------------|------------------|
| BC4001 | UKM |
|---------------------|------------------|
| NULL | ItalyZ |
|---------------------|------------------|
| NULL | ItalyZ |
|---------------------|------------------|
| NULL | UKY |
|---------------------|------------------|
| NULL | UKM |
|---------------------|------------------|
Here is my expected output :
|---------------------|------------------|
| ID | Geography |
|---------------------|------------------|
| AB0000 | ItalyZ |
|---------------------|------------------|
| AB4041 | ItalyB |
|---------------------|------------------|
| BC0000 | UKY |
|---------------------|------------------|
| BC4001 | UKM |
|---------------------|------------------|
| AB0001 | ItalyZ |
|---------------------|------------------|
| AB0001 | ItalyZ |
|---------------------|------------------|
| AB4042 | UKY |
|---------------------|------------------|
| BC0001 | UKM |
|---------------------|------------------|
I have been trying many various ways and trying to adapt running total solutions. I have also been trying to break apart my file in four different ones in order not to have an If function alternating between cases, and thus making it simpler, like this in my power query:
#"Added Custom2" = Table.AddColumn(#"Reordered Columns", "Sum", each if [Geography] = "UKM" then [Number AB range below 4000] + 1
else if [Geography] = "UKY" then [Number AB range above 4000] + 1
else if [Geography] = "ItalyB" then [Number BC range above 5000]
else [Number BC range below 5000] + 1)
But absolutely nothing works. This maddening.
I will answer a further simplified problem as I don't want to address the ID letter prefixes.
Let's say we have the following table (I've included :
ID, Group
-----------
0, A
1, A
300, B
525, C
null, A
null, B
null, B
null, C
And want to generate a new column NewID that will replace ID.
ID, Group, NewID
------------------
0, A, 0
1, A, 1
300, B, 300
525, C, 525
null, A, 2
null, B, 301
null, B, 302
null, C, 526
Here's a method that uses Table.AddIndexColumn:
let
Source = <First Table Above>,
#"Grouped Rows" = Table.Group(Source, {"Group"}, {{"ID", each List.Max([ID]), type number}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", (C) => Table.AddIndexColumn(Table.SelectRows(Source, each _[Group] = C[Group]),"NewID",C[ID],1)),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"NewID"}, {"NewID"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Custom",{"ID"})
in
#"Removed Columns"
First, we group by Group to find the maximal ID per Group:
Then we add a new column where each row in the column is a table defined by filtering the original table down to just the current group and then adding an index column starting at the maximal ID we just found. This is the most complex step.
From here, we expand the Custom table column (choosing the column we don't already have) and remove the old ID column. We're now where we need to be short of any sorting or column typing we choose to do.
Edit: I've made a mistake in the above. Note that the NewID for Group A is 1,2,3 instead of 0,1,2 I was trying for.
To fix this for this simple example, you can use List.Min instead of List.Max in the group-by step.
For a more complex example, you may need to add an index column to the source table so you can merge back to it after expanding and only use the new NewID for previously null ID values since we can't guarantee they are contiguous.
Here's the code:
let
Source = <First Table Above>,
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Grouped Rows" = Table.Group(#"Added Index", {"Group"}, {{"ID", each List.Max([ID]), type number}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", (C) => Table.AddIndexColumn(Table.SelectRows(Table.Sort(#"Added Index",{"ID"}), each _[Group] = C[Group]),"NewID",C[ID]+1,1)),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Index", "NewID"}, {"Index", "NewID"}),
#"Merged Queries" = Table.NestedJoin(#"Added Index", {"Index"}, #"Expanded Custom", {"Index"}, "Expanded Custom", JoinKind.LeftOuter),
#"Expanded Expanded Custom" = Table.ExpandTableColumn(#"Merged Queries", "Expanded Custom", {"NewID"}, {"NewID"}),
#"Added Custom1" = Table.AddColumn(#"Expanded Expanded Custom", "ReplaceID", each if [ID] = null then [NewID] else [ID]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom1",{"ID", "NewID"})
in
#"Removed Columns"
The complex step changes just a bit:
(C) => Table.AddIndexColumn(
Table.SelectRows(
Table.Sort(#"Added Index", {"ID"}),
each _[Group] = C[Group]
),
"NewID", C[ID] + 1, 1
)
The difference is that we need to add a sort so the nulls come after all the already assigned ID values and start indexing the nulls at C[ID] + 1 instead of just C[ID].
Here's a version that has fewer steps (no group by, expand, or merge) but a bit more complicated function:
let
Source = <First Table Above>,
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index", "Custom", (C) => Table.SelectRows(#"Added Index", each _[Group] = C[Group])),
#"Added NewID" = Table.AddColumn(#"Added Custom", "NewID", (C) => if C[ID] = null then Table.SelectRows(Table.AddIndexColumn(Table.SelectRows(C[Custom], each _[ID] = null), "NewID", List.Max(C[Custom][ID])+1,1), each _[Index] = C[Index]){0}[NewID] else C[ID]),
#"Removed Columns" = Table.RemoveColumns(#"Added NewID",{"Custom"})
in
#"Removed Columns"
The first added Custom column is just the indexed source table filtered to the current Group. Then we add the NewID column defined as:
(Read from the inside out.)
(C) =>
if C[ID] = null
then Table.SelectRows(
Table.AddIndexColumn(
Table.SelectRows(C[Custom], each _[ID] = null),
"NewID", List.Max(C[Custom][ID]) + 1, 1
),
each _[Index] = C[Index]
){0}[NewID]
else C[ID]
Similar to before, we take the group subtable Custom, just pick the null ID rows and index them starting from the max non-null ID plus one. This still leaves us with a table, so we just want the row in this subtable that corresponds to the Index from the whole table. We use {0}[NewID] to extract the value from the cell in the first (only) row in the table in the [NewID] column. For non-null ID values, the else clause just leaves them as they were.
As in my other answer, here's a simplified problem ignoring the ID letter prefixes you have.
ID, Group | NewID
-----------|------
4, A | 4
7, A | 7
300, B | 300
525, C | 525
null, A | 10
9, A | 9
null, A | 11
null, B | 301
null, C | 526
null, A | 12
null, B | 302
Starting from the left part of the table, we want to compute the new column NewID.
In this answer, I will write a custom function that's written recursively using the List.Generate function.
From the linked documentation the function is set up like this
List.Generate(
initial as function, /*Set all your initial variables*/
condition as function, /*Stopping criteria.*/
next as function, /*Define how to update at each step.*/
optional selector as nullable function /*Pick output element.*/
) as list
Define a function that takes a column potentially containing nulls and fills the nulls incrementally from the maximal non-null value:
(Column as list) as list =>
let
Generate =
List.Generate(
() => [x = Column{0}, i = 0, n = List.Max(Column)],
each [i] < List.Count(Column),
each [
i = [i] + 1,
x = if Column{i} = null then [n] + 1 else Column{i},
n = if Column{i} = null then [n] + 1 else [n]
],
each [x]
)
in
Generate
When you define the function, it looks like this and can be re-used in any other queries:
You can use it by choosing an existing column in an existing table and clicking the Invoke button.
This will create a new list in your Queries pane named Invoked Function that is that function applied to the column you selected.
You can also create a blank query and pass it a list. For example, FilterNulls({4,7,null,9,null,null}) returns {4,7,10,9,11,12}.
Here's what that looks like in the query editor.
What we really want to do is use this function as a column transformation in a group by operation and then expand:
let
Source = <Data Table Source Here>,
#"Grouped Rows" = Table.Group(Source, {"Group"}, {{"FillNulls", each FillNulls([ID]), type list}}),
#"Expanded FillNulls" = Table.ExpandListColumn(#"Grouped Rows", "FillNulls")
in
#"Expanded FillNulls"
Here's what it looks like after grouping but before expanding:
Notice what the function is doing. We're applying the function FillNulls on the ID column for each separate Group.
This is a similar amount of steps and complexity as the other answer but uses a function that is constructed in a recursive way that may be more familiar to you.
I am having some issues with how to approach my query so any help would be greatly appreciated.
I have a date column that I need to increase based on two other columns values.
e.g. Date Reported column - 17/12/2018
If my Impact Column = "Urgent" and my Department = "Stores" I would need to increase my Date Reported Column to 18/12/2018
However if my Impact Column = "Standard" and my Department = "Floor" I would need to increase my Date Reported Column to 20/12/208
I would ideally like to not touch the original Date Reported Column but move this new value to another column.
So Far I have created a custom column and this is my code however it doesnt work.
AmendedDateReported = if(And(SurveyCorrectiveAction[Impact] = "Urgent", SurveyCorrectiveAction[LookUp] = "Stores"), Date.AddDays([DateReported],1),Blank ())
Thanks
Paula
Updated code, The formula seems to be pulling ok but the date part wont update:
#"Sorted Rows" = Table.Sort(Source,{{"DateReported", Order.Ascending}}),
#"Changed Type" = Table.TransformColumnTypes(#"Sorted Rows",{{"DateReported", type date}}),
#"Sorted Rows1" = Table.Sort(#"Changed Type",{{"DateReported", Order.Descending}}),
#"Added Custom" = Table.AddColumn(#"Sorted Rows1", "Date Repaired", each ""),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{{"Date Repaired", type text}}),
#"Duplicated Column" = Table.DuplicateColumn(#"Changed Type1", "DateReported", "DateReported - Copy"),
#"Renamed Columns" = Table.RenameColumns(#"Duplicated Column",{{"DateReported - Copy", "AmendedDateReported"}}),
#"Merged Amendments" = Table.NestedJoin(#"Renamed Columns",{"Impact", "Department"},TLU_FaultTimeScales,{"Impact", "Department"},"TLU_FaultTimeScales",JoinKind.LeftOuter),
#"Expanded Amendments" = Table.ExpandTableColumn(#"Merged Amendments", "TLU_FaultTimeScales", {"Amendment Day"}, {"Amendment Day"}),
AmendedDateReported = Table.AddColumn(#"Expanded Amendments", "AmendedDateReported", each try Date.AddDays([DateReported],[Amendment Day]) otherwise [DateReported], type date)
in
#"Renamed Columns"
You could try:
AmendedDateReported =
Table.AddColumn(
#"Previous Step",
"Amended Date Reported",
each Date.AddDays(
[Date Reported],
if [Impact] = "Urgent" and [Department] = "Stores" then 1
else if [Impact] = "Standard" and [Department] = "Floor" then 3
else 0
),
type date
)
If you have several combinations of Impact / Department which have variable effect on amending the date, it would make more sense to put those in a separate table:
+----------+------------+----------------+
| Impact | Department | Amendment Days |
+----------+------------+----------------+
| Urgent | Stores | 1 |
| Standard | Floor | 3 |
+----------+------------+----------------+
You can then join this table to retrieve the amendment days:
#"Merged Amendments" = Table.NestedJoin(#"Previous Step",{"Impact", "Department"},tblAmendments,{"Impact", "Department"},"tblAmendments",JoinKind.LeftOuter),
#"Expanded Amendments" = Table.ExpandTableColumn(#"Merged Amendments", "tblAmendments", {"Amendment Days"}, {"Amendment Days"}),
AmendedDateReported = Table.AddColumn(#"Expanded Amendments", "Amended Date Reported", each try Date.AddDays([Date Reported],[Amendment Days]) otherwise [Date Reported], type date)
in
AmendedDateReported
Remember to update the final variable name after the in clause.
Say I am importing something like the following into PowerBI:
Date | Quantity
|---------------------|------------------|
| 1/1/2018 | 22 |
| 1/3/2018 | 30 |
| 1/4/2018 | 10 |
|---------------------|------------------|
Where the external source table is a series of date, value rows with some date values missing. I'd like to execute some DAX/M to add any missing date rows into the data set, where 'Quantity' value is taken from the first date prior. So my resulting dataset would like like this:
Date | Quantity
|---------------------|------------------|
| 1/1/2018 | 22 |
| 1/2/2018 | 22 |
| 1/3/2018 | 30 |
| 1/4/2018 | 10 |
|---------------------|------------------|
Can this be done in PowerBI?
Help is much appreciated!
You can do this in DAX by creating a new table with all of the dates in your range as follows:
FullTable =
ADDCOLUMNS(
CALENDAR(MIN(Table1[Date]), MAX(Table1[Date])),
"Quantity",
LOOKUPVALUE(
Table1[Quantity],
Table1[Date],
MAXX(
FILTER(Table1, Table1[Date] <= EARLIER([Date])),
[Date]
)
)
)
The CALENDAR function gives you a range of dates from you minimum to maximum dates in your original table. From there, we add a new column, Quantity, and define it as the value we get when looking up the Quantity in the original table for the date that was the maximum date occurring on or before the date in the current row.
In Power Query "M" code, see the comments to understand the different steps. Basically, it generates a new table with all the days, takes the Quantity values if avaliable and fill down the gaps.
Hope this help you.
let
Source = Excel.CurrentWorkbook(){[Name="Test"]}[Content],
Converted = Table.TransformColumnTypes(Source,{{"Date", type date}}),
// force conversion to date type to avoid problems if date values comes from an Excel table (in Excel they are numbers).
DatesCol = Converted[Date],
// list from Date column of Converted table
Ini = List.Min(DatesCol),
End = List.Max(DatesCol),
Size = Number.From(End - Ini) +1,
DatesNew = List.Dates(Ini, Size, #duration(1, 0, 0, 0)),
// generates a new list of dates from min to max
NewTable = Table.FromColumns({DatesNew},{"Dates"}),
// new table from new list of dates, column name is "Dates" not "Date".
Joined = Table.Join(NewTable,"Dates",Converted,"Date",JoinKind.FullOuter),
// join both table NewTable and Converted (from Source) and keep all the dates from the DatesNew list.
Removed = Table.RemoveColumns(Joined, "Date"),
// remove the original Date column as it's not needed.
Result = Table.FillDown(Removed,{"Quantity"})
// Fill Down the Quantity column.
in
Result
Two items below should do it
// Query name fnGeT
(TableName as table, DateSearch as date) =>
let Source2 = Table.SelectRows(TableName, each [Date] < DateSearch and [quantity] <> null),
Max= Table.Group(Source2, {}, {{"Max", each List.Max([Date]), type date}}),
MaxDate = Table.SelectRows(Source2, each [Date] = Max{0}[Max]),
Value = MaxDate{0}[quantity]
in Value
and
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Base = Table.TransformColumnTypes(Source,{{"Date", type date}, {"quantity", Int64.Type}}),
// Generate list of dates between Max and Min dates of Table1
DateRange = Table.Group(Base, {}, {{"MinDate", each List.Min([Date]), type date}, {"MaxDate", each List.Max([Date]), type date}}),
StartDate = DateRange[MinDate]{0},
EndDate = DateRange[MaxDate]{0},
List ={Number.From(StartDate)..Number.From(EndDate)},
#"Converted to Table" = Table.FromList(List, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
FullList = Table.TransformColumnTypes(#"Converted to Table",{{"Column1", type date}}),
//Right Anti Join to find dates not in original Table1
#"Merged Queries" = Table.NestedJoin(Base,{"Date"},FullList,{"Column1"},"Table2",JoinKind.RightAnti),
#"Removed Other Columns" = Table.SelectColumns(#"Merged Queries",{"Table2"}),
Extras = Table.ExpandTableColumn(#"Removed Other Columns", "Table2", {"Column1"}, {"Date"}),
Combined = Base & Extras,
#"Added Custom" = Table.AddColumn(Combined, "Custom", each if [quantity]<>null then [quantity] else fnGet(Combined,[Date])),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"quantity"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Custom", "quantity"}})
in #"Renamed Columns"