Trying to create a new query from the existing "Master" Query using below formula:
let
Source = Table.SelectColumns('Original Source Name',{'Column Name','Column Name2'})
in
Source
which works fine, however I am looking to see if there is any other formula which would do the same but in a way that it will create the new query with a range of columns , for example Column 30- 67 ( in this case when the original Excel file is updated, inserting a column in this range it would automatically update in the PBI too when refreshed)
Here's one possible way. If you start with this table, named Table1:
You can reference it in a new query like this:
let
Source = Table.SelectColumns(Table1, List.Range(Table.ColumnNames(Table1), 2, 3))
in
Source
...to get this:
The formula selects a range of columns from the table starting at the column at index position 2, and spanning 3 columns. (The index starts with 0.) For columns 30-67, you would change the 2 to 31 and the 3 to 37. You would change Table1 to your Original Source Name as well.
See these links for more info on List.Range and Table.ColumnNames.
Related
I have a query in Power BI that takes two parameter: Start Date and End Date.
Whenever I pass these Dates it return a table of Date that contain few columns created according to this range of date such as Date, QuarterofYear, Year, MonthName......etc.
Can we create a mapping data flow in ADF that takes two parameter as input and return a calculated table according to provided dates?
Is there any function that return the range of dates?
For your request: "I want that I pass two date Start Date and End Date in ADF Mapping Data Flow , and Data flow will Create a column such as "Date" that contain that number of Date rows. Is there any function for this? Exam. Start Date=20-01-2019, End Date=20-01-2020 Then Date Column Values should be: 20-01-2019 21-01-2019 ......... ......... 20-02-2020", according the Data Factory documents and my experience, the answer is no, we can't achieve it in Data Flow.
There is a solution to this, but it is a bit tricky.
TL;DR
The general data flow looks like this:
We need a dummy source with exactly one row which contains whatever.
Then we derive a column where we use the mapLoop() expression to create an array of all the dates we want to get rows for.
Finally, we need to flatten the array column which will result in one row per array entry and thus one row per date.
Walkthrough
Source dummy
Each dataflow needs a source and we need exactly one row to make our dataflow work. To achieve this I've created a dataset called empty of type CSV in my data lake which has this content:
empty
""
This is our source definition:
And its result looks like this:
Derived column days
This is where the magic happens!
We create a new column dates which is an array of all the dates we want to have in our date table:
In this scenario we want a date table starting on 2019-01-01 and reaching one year into the future. The full expression looks like this:
mapLoop(
addDays(currentDate(), 365) - toDate(2019-01-01),
addDays(toDate(2019-01-01), #index)
)
This is what happens here:
the mapLoop() function builds an array of elements. You specify the number of elements you want to have and the lambda expression to calculate each of the elements. For example, mapIndex([1, 2, 3, 4], #item + 2 + #index) results in [4, 6, 8, 10]
addDays(currentDate(), 365) - toDate('2019-01-01') is the number of days between our start (2019-01-01) and end date (1 year in the future from now) and thus the number of dates we want to have in our resulting array.
addDays(toDate(2019-01-01), #index) calculates each array item by adding #index days to our start date. This is executed for the number of days we've calculated before and #index is the array position. Thus, the first element of the array will be 2019-01-01 + 1, the second 2019-01-01 + 2 and so on.
Our stream now has these columns:
Flatten
Finally, you need a flatten transformation which will expand each item in your array to its dedicated row. We can also dismiss the useless empty column in this step:
And this finally results in what we wanted to achieve:
References
Data transformation expressions in mapping data flow
I want to move a single value from column B to column A, how can I achieve it in the most simplest way in Power Query / Query Editor (Power BI)?
Please see attached images.
I know I might need to declare a variable so please enlighten me. By the way, I will delete row 1 afterwards, promote my headers, and rename column2 as PERIOD.
Thank you.
This might be along the lines of what you want to do.
If I start with this table named as Table1:
Then I click on the fx to the left of the formula bar:
And type = Table.InsertRows(Source, Table.RowCount(Source), {[Column2 = Source[KP20 rate]{0}, KP20 rate = null, Column4 = null]}) into the formula bar:
I used Table.InsertRows to create a new row in Table1. Source is the name of the latest state of Table1 after it is pulled into Power Query and before I do this step. So I actually use Source as the name of the table for this step instead of Table1. (Each applied step basically results in its own table. You probably know this already, but others may not.) So for this step I use Source as the table name in the Table.InsertRows statement. Then, since I want the new row to appear at the bottom of Source, I just enter the Table.RowCount of Source as the row number location for the new row. Then I enter each of the Columns' names and their values to be added. For Column2, I entered the value "Source[KP20 rate]{0}." Source[KP20 rate]{0} basically treats column KP20 rate as a list, where {0} serves as a pointer to the first item in the list. To target the second item in Source[KP20 rate] you would use Source[KP20 rate]{1}. You can see that I set the values for the other two columns (KP20 rate and Column4) to null.
The result:
Here's the M code in case you want to see it:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Custom1 = Table.InsertRows(Source, Table.RowCount(Source), {[Column2 = Source[KP20 rate]{0}, KP20 rate = null, Column4 = null]})
in
Custom1
The picture I have attached shows what my power query table looks like (exactly the same as source file) and then underneath what I would like the final end product to look like.
Correct me if I'm wrong but I thought the purpose of power query/power bi was to not manipulate the source file but do this in power query/power bi?
If that's the case, how can I enter new columns and data to the existing table below?
You can add custom columns without manipulating source file in power bi. Please refer to below link.
https://learn.microsoft.com/en-us/power-bi/desktop-add-custom-column
EDIT: Based on your comment editing my answer - Not sure if this helps.
Click on edit queries after loading source file to power bi.
Using 'Enter Data' button entered sample data you provided and created new table. Data can be copy pasted from excel. You can enter new rows manually. Using Tag number column to keep reference.
Merge Queries - Once the above table is created merged it with original table on tag number column.
Expand Table - In the original table expand the merged table. Uncheck tag number(as it is already present) and uncheck use original column name as prefix.
Now the table will look like the way you wanted it.
You can always change data(add new columns/rows) manually in new table by clicking on gear button next to source.
Here is the closest solution to what I found from "manual data entry" letting you as much freedom as you would like to add rows of data, if the columns that you want to create do not follow a specific pattern.
I used an example for the column "Mob". I have not exactly reproduced the content of your cells but I hope that this will not be an issue to understand the logic.
Here is the data I am starting with:
Here is the Power Query in which I "manually" add a row:
#"Added Conditional Column" = Table.AddColumn(#"Changed Type", "Mob", each if [Tag Number] = "v" then null else null),
NewRows = Table.InsertRows(#"Added Conditional Column", 2, {[Mob="15-OHIO", Tag Number="4353654", Electronic ID=1.5, NLIS="", Date="31/05/2015", Live Weight="6", Draft="", Condition store="", Weighing Type="WEAN"]})
in
NewRows
1) I first created a column with only null values:
#"Added Conditional Column" = Table.AddColumn(#"Changed Type", "Mob", each if [Tag Number] = "v" then null else null),
2) With the "Table.InsertRows" function:
I indicated the specific line: 2, (knowing that power Bi start counting at zero, at the "headers" so it will the third line in the file)
I indicated the column at which I wanted to insert the value, i.e "Mob"
I indicated the value that all other other rows should have:
NewRows = Table.InsertRows(#"Added Conditional Column", 2, {[Mob="15-OHIO", Tag Number="4353654", Electronic ID=1.5, NLIS="", Date="31/05/2015", Live Weight="6", Draft="", Condition store="", Weighing Type="WEAN"]})
Here is the result:
I hope this helps.
You can apply this logic for all the other rows.
I do not think that this is very scalable however, becaue you have to indicate each time the values of the rows in the other columns as well. There might be a better option.
I have my data source something like below.
I need to show output in the report as below.
I tried using the unpivot column and getting something like this, how to count the occurrence value of each Business value.
Plot following mesure against Value column (from your unpivot table):
Business Occurance = COUNTROWS('your unpivot table')
We have to remove the Attribute column as the next step to Unpivot. Then my table should be looks like this.
Now create a new table with following Dax function, let's say the current table as Business Data (Your Unpivot table)
Occurrence Table = DISTINCT('Business Data')
Now end result table should look like this,
You can make use of this table for your table visual in the report.
Note: You can add n-number of rows and column into your source table and this logic will do magic to get the correct result.
I have marked two places first marked place you have to add Value column then click second marked place one dropdown value is open click count menu
I am fairly new to powerbi and I need your help in one task on which I am stuck on.
Basically I have two tables and I need to compare the value from table one with a row of table 2 and return the output.
Table 1
I need to compare values in column a & b and get a match from table 2.
For example if row 1 has BY Green & BS HIGH then I need to check this value from matrix table below and return the output in column value as either 0 or 1.
Table 2
As you can see the Table 2 first row has value BY Green and BS low has a value '0'
Try this...
Index() returns a value from the matrix (in purple) based upon the intersections of the two match()'s. The first is the Vertical match in from the Table1:Col A; the second is the Horizontal match from table1:Col B. The value found at that intersection is returned.
... My apologies ... just saw this was a BI request... no worries...
First, Need fixup table2 as a lookup file:
First, click a cell in table 2 (don't edit), then Data menu >frm table/range, will bring up the Power Query window. Select columns B (not A) through Col F), then in the PQ Transform menu > Unpivot to create the new lookup table. this can either be saved as a new table or be used by reference.
Next, open and merge Table 1 PQ_Table 2 (Be sure to select BOTH Columns in BOTH Tables, in the same order). Then, expand the table tab following the merge expand the table tab. I only selected the value to return but you can return all the values to verify, then delete the unneeded columns.
Hope this helps...
Good Luck.