Is it possible to refer from Power Query (M) to DAX calculated table? I would like to get DAX table as a source to my power query.
The purpose. I have grouping table made in DAX. I would like to make econometric model with R. So I would like to transform the DAX table with R to get the model parameters. I would like to use these parameters further in DAX measures (not just display them).
Currently I dump the DAX grouping table to Excel file and then pull it up with Power Query.
Actually, there is a way.
DISCLAIMER: This is a hack. You should not rely on this way.
1. Create DAX calculated table
Input any DAX formula that evaluates to a table in Modeling > New Table.
2. Check port number using DAX Studio
Connect to your PBI Desktop data model using DAX Studio, and check the port number where the data model is hosted. It should be displayed in the right bottom of the window.
3. Import the table to Power Query
Click Get Data > Analysis Services and input the address (in my example "localhost:50293") to Server. Then navigate to your DAX calculated table.
it's not possible to refer to a DAX calculated table in M as it's loaded into DAX/Power Pivot engine after M has done the transformations. You can't write to a DAX table after loading into R as well. You can do grouping in M, or if needed run R in the Power Query. One approach that I have used has to load the data, duplicate the query, run a group/filter on the new query, then use that data in a later stage in the report.
Hope that helps
Jonee is correct. This is not possible. DAX calculated tables are computed after the M queries have loaded and you cannot feed them back into Power Query without saving them externally like you are currently doing.
The M language is more powerful than you might think and very likely could do the same grouping operations, though depending on what they are, it might be fairly difficult. You can also use R or Python script within an M query if you are more comfortable with those.
Related
When I get data into Power BI I can edit the query as well as perform edit to the model.
What is difference between edit performed in query edit vs during modelling?
When you edit the query, you use Power Query, with its own Query Editor user interface. The steps you apply are recorded in the "M" language. Use Power Query to extract, transform, and finally load data into the Data Model.
Once the data is in the Data Model, you use DAX to create measures that you use in visuals. You can also use DAX to add more columns or even tables to the data model.
Whether to use Power Query or DAX to add columns or tables to the data model depends on a variety of factors. Some things are dead easy to do in Power Query, but harder to achieve with DAX, and vice versa. If you create a column with a formula that depends on a DAX measure, then you can only do that with DAX, because Power Query is not aware of the measures that are created after the load into the data model.
Power Query is very powerful, but the M code syntax is very different to the Excel formula syntax, or the VBA macro language. Learning to write advanced M code can be quite challenging.
DAX, on the other hand, behaves very similar to Excel formulas. Many Excel functions can even be used in DAX verbatim. If you know Excel, you've already got a head start on DAX and you can ease your way into it by learning additional functions and then expanding into more complex formulas.
The latter is probably the reason why many data manipulations are done in DAX, even though they could as well have been done in Power Query.
There are also some efficiencies with data storage and performance. Power Query makes use of query folding with SQL queries, for example, where its transformations are actually performed at the data source, i.e. on the SQL server side, and not in desktop client, and only the final query result is transferred to the desktop client.
Edit after comment: When the data is loaded into the data model, an algorithm processes the data and sorts it in a way that is most efficient for maximum compression and minimum storage. I don't have any concreate examples, but adding a column in Power Query will result in a smaller footprint than adding the same column with DAX. Read more about the compression algorithm VertiPaq here: https://towardsdatascience.com/inside-vertipaq-in-power-bi-compress-for-success-68b888d9d463
But apart from that, it mainly comes down to personal preference based on skill and experience.
By the way, many of your questions can be answered by reading through the Microsoft documentation, e.g. https://learn.microsoft.com/en-us/power-bi/guidance/import-modeling-data-reduction
I tried to use the mid and find function in power BI as it can be done in excel. However, I get the error 'find' wasn't recognized.
After searching for a while a have a conclusion that FIND and MID function work in DAX (Excel and Power BI - but not in M query (edit custom) column). Instead of using find, in and M query we should use BIText, PositionOfAny.
Here is an example:
DAX:
MID([TRAFFIC_SIGNAL]), find([TRAFFIC_SIGNAL],"&"),3)
M query:
Text.Combine({Text.Start(Text.Upper([TRAFFIC_SIGNAL]), 3), " ",
Text.Middle(Text.Upper([TRAFFIC_SIGNAL]),
Text.PositionOfAny([TRAFFIC_SIGNAL], {"&"})+1, 3)})
It works so I would like to share because I haven't know the difference between DAX and m query in Power BI before, but this example helps.
I am not sure what you are looking for, but I am just going to highlight the differences between m query and DAX.
M Query:
M query is used to bring the data into the model
This can be accessed using the Power Query Editor
DAX:
DAX is used to create measures and columns after the data is pulled
This is mainly used for summarizing the data
Hope this helps.
M and DAX are entirely different languages used for different purposes.
Primarily,
M is how you get and transform your data before loading to the data model.
DAX is for reading the data in your data model aggregating it to show in visuals.
There are plenty of things you can do with both where it isn't clear which is the better option. It can be highly case-dependent but the above is a simple guide.
In any case, I wouldn't recommend that M code for that purpose. Something like the following should be simpler and more similar to the DAX code:
Text.Middle([TRAFFIC_SIGNAL], Text.PositionOf([TRAFFIC_SIGNAL],"&"), 3)
Should we construct bridge tables with DAX or M?
Picture stolen from here
It seems very tempting to use DAX. With DAX the code is short and clear:
IDList = DISTINCT(
UNION(
DISTINCT(Table1[ID])
,DISTINCT(Table2[ID])
))
Moreover, DAX tables do not need to be loaded as M tables. However I wonder if advantage of DAX over M is not illusory? M seems to load once and DAX seems to be calculated on the fly, maybe anytime, over and over?
DAX calculated tables are re-calculated if any of the tables it pulls data from are refreshed or updated in any way. (from https://learn.microsoft.com/en-us/power-bi/desktop-calculated-tables )
They're not re-calculated "on the fly", nor "over and over". There's no difference to the refresh cycle of your Power BI data model, between using a DAX calculated table or an M query table. You may however find that DAX calculated tables refresh faster than M, depending on the complexity of the table...
Considering M Tables, M Conditional Columns, M Custom Columns, DAX Tables, DAX Calculated Columns and DAX Measures. It is only the DAX Measures that gets created on the fly and is not a part of the data model.
So for a Simple Bridge Table, DAX Table and M Table have no real advantage over each other.
Both Tables allows one to create relationships. Now, When I say simple Bridge Table, it is something that is created from 2 or 3 tables and uses the same column to establish relationships with two or more tables.
But when the requirements become complex and agile (growing over time), the maintenance and the developments efforts also increases, if it is created by DAX. (my personal opinion and I think most people's personal opinion as well.)
If it is created by M, then it is more easy to add a new column or filter based on a logic or to replace an existing value.
Taking back to the Thumb Rule :- If it is created by DAX, then M Cannot be used on top of it to make changes. So, if the Bridge Table is created by DAX, then it won't appear in the query editor and limits the advantages of GUI to make any required simple transformations in the data.
For a Simple Bridge Table :- DAX.
But For a complex and changing requirement :- M.
I am trying to get the first 4 digits from a string from a table in Power BI. The connection is a live connection / Direct which does not allow me to edit the query. Also, I am unable to create a new column. So I have to stick with creating a new Measure.
Now, I am using the following formula to get what I need.
LocationCd = mid(vw_DW_Contracts[ContractNumber],1,5)
but, this is not working a the vw_DW_Contracts table cannot be used in a measure. Is there a workaround to such problem?
I do not have access to the analysis service so cannot make any modifications in the source.
Please help.
Thanks
but, this is not working a the vw_DW_Contracts table cannot be used in a measure.
I'm not sure what you mean by this, but I'm guessing the message you see is telling you that measures expect an aggregation. The formula you posted would be great as a calculated column where it can be evaluated row by row. Measures are aggregations over multiple rows.
If you are trying to make a new field that is the location code that can be used in visuals on a categorical axis, this should be a column rather than a measure. You could write a measure to show a location cd using something like LASTNONBLANK (mid(vw_DW_Contracts[ContractNumber],1,5), 1) but I doubt that is what you want.
I have been working on Power BI for a while now and I often get confused when I browse through help topics of it. They often refer to the functions and formulas being used as DAX functions or Power Query, but I am unable to tell the difference between these two. Please guide me.
M and DAX are two completely different languages.
M is used in Power Query (a.k.a. Get & Transform in Excel 2016) and the query tool for Power BI Desktop. Its functions and syntax are very different from Excel worksheet functions. M is a mashup query language used to query a multitude of data sources. It contains commands to transform data and can return the results of the query and transformations to either an Excel table or the Excel or Power BI data model.
More information about M can be found here and using your favourite search engine.
DAX stands for Data Analysis eXpressions. DAX is the formula language used in Power Pivot and Power BI Desktop. DAX uses functions to work on data that is stored in tables. Some DAX functions are identical to Excel worksheet functions, but DAX has many more functions to summarize, slice and dice complex data scenarios.
There are many tutorials and learning resources for DAX if you know how to use a search engine. Or start here.
In essence: First you use Power Query (M) to query data sources, clean and load data. Then you use DAX to analyze the data in Power Pivot. Finally, you build pivot tables (Excel) or data visualisations with Power BI.
M is the first step of the process, getting data into the model.
(In PowerBI,) when you right-click on a dataset and select Edit Query, you're working in M (also called Power Query). There's a tip about this in the title bar of the edit window that says Power Query Editor. (but you have to know that M and PowerQuery are essentially the same thing). Also (obviously?) when you click the get data button, this generates M code for you.
DAX is used in the report pane of PowerBI desktop, and predominantly used to aggregate (slice and dice) the data, add measures etc.
There is a lot of cross over between the two languages (eg you can add columns and merge tables in both) - Some discussion on when to choose which is here and here
Think of Power Query / M as the ETL language that will be used to format and store your physical tables in Power BI and/or Excel. Then think of DAX as the language you will use after data is queried from the source, which you will then use to calculate totals, perform analysis, and do other functions.
M (Power Query): Query-Time Transformations to shape the data while you are extracting it
DAX: In-Memory Transformations to analyze data after you've extracted it
One other thing worth mentioning re performance optimisation is that you should "prune" your datatset (remove rows / remove columns) as far "upstream" - of the data processing sequence - as possible; this means such operations are better done in Power Query than DAX; some further advice from MS here: https://learn.microsoft.com/en-us/power-bi/power-bi-reports-performance