I have been working on Power BI for a while now and I often get confused when I browse through help topics of it. They often refer to the functions and formulas being used as DAX functions or Power Query, but I am unable to tell the difference between these two. Please guide me.
M and DAX are two completely different languages.
M is used in Power Query (a.k.a. Get & Transform in Excel 2016) and the query tool for Power BI Desktop. Its functions and syntax are very different from Excel worksheet functions. M is a mashup query language used to query a multitude of data sources. It contains commands to transform data and can return the results of the query and transformations to either an Excel table or the Excel or Power BI data model.
More information about M can be found here and using your favourite search engine.
DAX stands for Data Analysis eXpressions. DAX is the formula language used in Power Pivot and Power BI Desktop. DAX uses functions to work on data that is stored in tables. Some DAX functions are identical to Excel worksheet functions, but DAX has many more functions to summarize, slice and dice complex data scenarios.
There are many tutorials and learning resources for DAX if you know how to use a search engine. Or start here.
In essence: First you use Power Query (M) to query data sources, clean and load data. Then you use DAX to analyze the data in Power Pivot. Finally, you build pivot tables (Excel) or data visualisations with Power BI.
M is the first step of the process, getting data into the model.
(In PowerBI,) when you right-click on a dataset and select Edit Query, you're working in M (also called Power Query). There's a tip about this in the title bar of the edit window that says Power Query Editor. (but you have to know that M and PowerQuery are essentially the same thing). Also (obviously?) when you click the get data button, this generates M code for you.
DAX is used in the report pane of PowerBI desktop, and predominantly used to aggregate (slice and dice) the data, add measures etc.
There is a lot of cross over between the two languages (eg you can add columns and merge tables in both) - Some discussion on when to choose which is here and here
Think of Power Query / M as the ETL language that will be used to format and store your physical tables in Power BI and/or Excel. Then think of DAX as the language you will use after data is queried from the source, which you will then use to calculate totals, perform analysis, and do other functions.
M (Power Query): Query-Time Transformations to shape the data while you are extracting it
DAX: In-Memory Transformations to analyze data after you've extracted it
One other thing worth mentioning re performance optimisation is that you should "prune" your datatset (remove rows / remove columns) as far "upstream" - of the data processing sequence - as possible; this means such operations are better done in Power Query than DAX; some further advice from MS here: https://learn.microsoft.com/en-us/power-bi/power-bi-reports-performance
Related
Should we use the group by function in Power Query and create a new table, or is it better to create as many measures as we need ? (one measure for each column) ?
Which one is more powerful?
Thank you !
It depends on your purpose. If you have a granular fact table that you want to aggregate first before creating the data model, you can do that through Power Query before feeding the model. Even then, I would recommend doing it on the server-side if you are bringing a SQL table; so that you can perform a native SQL group by rather than having to do it through Power Query syntax solely. Power Query has some performance lagging and each nth step in PQ is evaluated from 1st step internally and it requires a full refresh of the table.
However, if you only want to perform group by to be utilized in an analysis, it is always a good idea to use DAX measures and refrain from using PQ. Also, you can't resort to PQ for different analysis scenarios. DAX is built for those scenarios and it is extremely powerful. DAX measures are the most powerful concept of Power BI. Also, they get evaluated in filter context/slicers; i.e. respond to the selection of values in slicers and / or whatever is present in the Axis (business case)
There are tons of supports for DAX measure optimization, such as SQLBI, Stack, Power BI community. If optimized correctly, DAX measures enhance report performance tremendously without creating any lagging in the report at all.
Few resources to look into
1
2
3
When you are creating a new table in power query, it means results are pre calculated and there will be some performance gain if we consider report usage. But, it will increase your Data Model size. Where as Measure will calculate things on the fly. This will keep your model size same but add some slowness in the presentation part. As a whole, there is no specific answer for your question as per my knowledge as it depends on so many other things like-
Your data size
How many measure you wants to create
How complex your logic inside measure's
How often you need reload your data
and so on...
When I get data into Power BI I can edit the query as well as perform edit to the model.
What is difference between edit performed in query edit vs during modelling?
When you edit the query, you use Power Query, with its own Query Editor user interface. The steps you apply are recorded in the "M" language. Use Power Query to extract, transform, and finally load data into the Data Model.
Once the data is in the Data Model, you use DAX to create measures that you use in visuals. You can also use DAX to add more columns or even tables to the data model.
Whether to use Power Query or DAX to add columns or tables to the data model depends on a variety of factors. Some things are dead easy to do in Power Query, but harder to achieve with DAX, and vice versa. If you create a column with a formula that depends on a DAX measure, then you can only do that with DAX, because Power Query is not aware of the measures that are created after the load into the data model.
Power Query is very powerful, but the M code syntax is very different to the Excel formula syntax, or the VBA macro language. Learning to write advanced M code can be quite challenging.
DAX, on the other hand, behaves very similar to Excel formulas. Many Excel functions can even be used in DAX verbatim. If you know Excel, you've already got a head start on DAX and you can ease your way into it by learning additional functions and then expanding into more complex formulas.
The latter is probably the reason why many data manipulations are done in DAX, even though they could as well have been done in Power Query.
There are also some efficiencies with data storage and performance. Power Query makes use of query folding with SQL queries, for example, where its transformations are actually performed at the data source, i.e. on the SQL server side, and not in desktop client, and only the final query result is transferred to the desktop client.
Edit after comment: When the data is loaded into the data model, an algorithm processes the data and sorts it in a way that is most efficient for maximum compression and minimum storage. I don't have any concreate examples, but adding a column in Power Query will result in a smaller footprint than adding the same column with DAX. Read more about the compression algorithm VertiPaq here: https://towardsdatascience.com/inside-vertipaq-in-power-bi-compress-for-success-68b888d9d463
But apart from that, it mainly comes down to personal preference based on skill and experience.
By the way, many of your questions can be answered by reading through the Microsoft documentation, e.g. https://learn.microsoft.com/en-us/power-bi/guidance/import-modeling-data-reduction
Is it possible to refer from Power Query (M) to DAX calculated table? I would like to get DAX table as a source to my power query.
The purpose. I have grouping table made in DAX. I would like to make econometric model with R. So I would like to transform the DAX table with R to get the model parameters. I would like to use these parameters further in DAX measures (not just display them).
Currently I dump the DAX grouping table to Excel file and then pull it up with Power Query.
Actually, there is a way.
DISCLAIMER: This is a hack. You should not rely on this way.
1. Create DAX calculated table
Input any DAX formula that evaluates to a table in Modeling > New Table.
2. Check port number using DAX Studio
Connect to your PBI Desktop data model using DAX Studio, and check the port number where the data model is hosted. It should be displayed in the right bottom of the window.
3. Import the table to Power Query
Click Get Data > Analysis Services and input the address (in my example "localhost:50293") to Server. Then navigate to your DAX calculated table.
it's not possible to refer to a DAX calculated table in M as it's loaded into DAX/Power Pivot engine after M has done the transformations. You can't write to a DAX table after loading into R as well. You can do grouping in M, or if needed run R in the Power Query. One approach that I have used has to load the data, duplicate the query, run a group/filter on the new query, then use that data in a later stage in the report.
Hope that helps
Jonee is correct. This is not possible. DAX calculated tables are computed after the M queries have loaded and you cannot feed them back into Power Query without saving them externally like you are currently doing.
The M language is more powerful than you might think and very likely could do the same grouping operations, though depending on what they are, it might be fairly difficult. You can also use R or Python script within an M query if you are more comfortable with those.
I am new to Power BI and with the limited time given, I am stuck at how to come up with:
Below Table B-Row1 ("1/20" and "M"-Monday cell) - how to
specifically place the date measures in their specific cell and put
it in one column?
How can I merge the cells under the Total column?
How to add all the numbers from the Type1 and Type2 columns and place it in the merged cell in #2?
Any clues/direction/links on how to achieve the Target Table B below will be much appreciated.
PS. Below Table A. Current is just using Matrix Visualization in Power BI.
You can't exactly do what you are after. PowerBI allows you to rapidly put amazing visuals together however that comes at the price of lack of (easy) flexibility. You could build your own custom visual or look in App Source for a visual that does this, or build the Visual in some other tool (via custom code).
However, I'd recommend sticking with the PowerBI matrix, which will give you a cascading drill down and work out how best to align your data to it and other out of the box visuals. Once you start to delve in to convoluted work-arounds to give users data in exactly the format they request you start to burn a lot of time. Look for alternatives to tell the data's story and work with your end-user to buy in to it.
Just wanna share that I have resolved my problem not using one type of visualization, but through using 3 different visualizations in Power BI. I used:
1 Table visual for Date column
1 Table visual for Total column
1 Matrix visual for the Code+Type mapping and counts
I also used DAX function to get the Date format and another DAX function used for both Total and Code+Type counts(to filter data according to the specified date).
Thanks for the response, #Murray and #RADO.
I tried to use the mid and find function in power BI as it can be done in excel. However, I get the error 'find' wasn't recognized.
After searching for a while a have a conclusion that FIND and MID function work in DAX (Excel and Power BI - but not in M query (edit custom) column). Instead of using find, in and M query we should use BIText, PositionOfAny.
Here is an example:
DAX:
MID([TRAFFIC_SIGNAL]), find([TRAFFIC_SIGNAL],"&"),3)
M query:
Text.Combine({Text.Start(Text.Upper([TRAFFIC_SIGNAL]), 3), " ",
Text.Middle(Text.Upper([TRAFFIC_SIGNAL]),
Text.PositionOfAny([TRAFFIC_SIGNAL], {"&"})+1, 3)})
It works so I would like to share because I haven't know the difference between DAX and m query in Power BI before, but this example helps.
I am not sure what you are looking for, but I am just going to highlight the differences between m query and DAX.
M Query:
M query is used to bring the data into the model
This can be accessed using the Power Query Editor
DAX:
DAX is used to create measures and columns after the data is pulled
This is mainly used for summarizing the data
Hope this helps.
M and DAX are entirely different languages used for different purposes.
Primarily,
M is how you get and transform your data before loading to the data model.
DAX is for reading the data in your data model aggregating it to show in visuals.
There are plenty of things you can do with both where it isn't clear which is the better option. It can be highly case-dependent but the above is a simple guide.
In any case, I wouldn't recommend that M code for that purpose. Something like the following should be simpler and more similar to the DAX code:
Text.Middle([TRAFFIC_SIGNAL], Text.PositionOf([TRAFFIC_SIGNAL],"&"), 3)