I'm a beginner in Power BI development.
I have a table which looks like this, it has 447 rows:
How do I configure my capabilites.json file to get the data from this table?
I have two GroupingOrMeasure data fields in dataRoles and I limit each to be "max": 1.
The closest I got to get the table's data, is this property, which only returns the 'name' column values correctly:
"dataViewMappings": [
{
"categorical": {
"categories": {
"for": { "in": "name-col" }
},
"values": {
"select": [
{ "bind": { "to": "imports-col" } }
]
}
}
}
]
But with this I don't get the 'imports' column's values, the dataView.categorical.categories has only one array.
My goal is to get both the 'name' and 'imports' column's value, the 'imports' column having multiple values associated to one value of the 'name' column's.
In most cases that I tried, that would return both columns, each source had far fewer values (around 83 each) and the 'name' source having each value listed only once.
Found the solution:
"table": {
"rows": {
"select": [
{ "for": { "in": "name-col" } },
{ "for": { "in": "imports-col" } }
]
}
}
Correctly returns each row.
Related
I'm developing a Power Bi custom visual, but I have a problem: when the user adds dimensions to the visual, the order shown in the UI does not reflect the actual order of the columns in the data I get from Power Bi. See for example this screenshot:
This is very limiting in a lot of scenarios, for example if I want to draw a table with columns in the order that the user sets.
Why does the API behave like this? Doesn't seem logical to me, am I doing something wrong? Here is the data binding definition if it helps:
"dataRoles": [
{
"displayName": "Fields",
"name": "fields",
"kind": "Grouping"
},
{
"displayName": "Measures",
"name": "measures",
"kind": "Measure"
}
],
"dataViewMappings": [
{
"table": {
"rows": {
"select": [
{
"for": {
"in": "fields"
}
},
{
"for":{
"in": "measures"
}
}
],
"dataReductionAlgorithm": {
"window": {
"count": 30000
}
}
}
}
}
]
I think I solved it.
You can get the actual column order like this:
dataView.metadata.columns.forEach(col => {
let columnOrder = col['rolesIndex'].XXX[0];
});
where XXX is the name of the Data Role, which in the example in my question would be fields.
Note that you have to access the rolesIndex property by key name like I did above (or alternatively cast to any before accessing it), because the DataViewMetadataColumn type is missing that property for some reason.
I am not sure come up with the CREATE TABLE statement for below mentioned JSON data. I checked the supported deserializer libraries in Athena but I was not able to figure it out. Can someone please advise.
{
"1": [
{
"a": {
"id_info": [
{
"id": "a1",
"id_type": "a1fx",
"mt": 0,
"pv": 1
}
]
},
"b": {
"id_info": [
{
"id": "b1",
"id_type": "b1fx",
"mt": 0,
"pv": 1
}
]
}
}
]
}
My expected output from the data when I run SELECT query is
key,category,id,id_type,mt,pv
1,a,a1,a1fx,0,1
1,b,b1,b1fx,0,1
[Hi, there]
I want to create an ES query that only retrieves certain elements that match in the list.
Here is my ES index schema.
"test-es-2018":{
"aliases": {},
"mappings": {
"test-1": {
"properties": {
"categoryName": {
"type": "keyword",
"index": false
},
"genDate": {
"type": "date"
},
"docList": {
"properties": {
"rank": {
"type": "integer",
"index": false
},
"doc-info": {
"properties": {
"docId": {
"type": "keyword"
},
"docName": {
"type": "keyword",
"index": false
},
}
}
}
},
"categoryId": {
"type": "keyword"
},
}
}
}
}
There are documents listed in the category. Documents in the list have their own information.
*search query in Kibana.
source": {
"categoryName" : "food" ,
"genDate" : 1577981646638,
"docList" [
{
"rank": 2,
"doc-info": {...}
},
{
"rank": 1,
"doc-info": {...}
},
{
"rank": 5,
"doc-info": {...}
},
],
"categoryId": "201"
}
First, I want to get only the element value that match in the list.
I would like to see only documents with rank 1 in the list. However, if I query using match as below, the result is the same as *search query in kibana.
*match query in Kibana.
GET test-es-2018/_search
{
"query": {
"bool": {
"must": [
{ "match": { "docList.rank": 1 } },
]
}
}
}
In my opinion, it seems to print the entire list because it contains a document with rank one.
What I want is:
source": {
"categoryName" : "food" ,
"genDate" : 1577981646638,
"docList" [
{
"rank": 1,
"doc-info": {...}
},
],
"categoryId": "201"
}
Is this possible?
Second, I want to sort the docList by rank. I tried sorting by creating a query like the following, but it was not sorted.
*sort query in Kibana.
GET test-es-2018/_search?
{
"query" : {
"bool" : {...}
},
"sort" : [
{
"docList.rank" : {
"order" : "asc"
}
}
]
}
What I want is:
source": {
"categoryName" : "food" ,
"genDate" : 1577981646638,
"docList" [
{
"rank": 1,
"doc-info": {...}
},
{
"rank": 2,
"doc-info": {...}
},
{
"rank": 5,
"doc-info": {...}
},
],
"categoryId": "201"
}
I do not know how to access the list. Is there a good idea for both of these issues?
In general you could use source filter to retrieve only part of the document but this way it's not possible to exclude some fields based on their values.
As far as I know Elasticsearch doesn't support changing order of field values in the _source. Partly the desired result can be achieved by using nested fields along with inner_hits -> sort query expression. This way sorted subhits will be returned in the inner_hits section of the response.
P.S. Typically working with Elasticsearch you should consider indexed document as the smallest indivisible search unit.
I have bunch of documents like this:
{
"schema": "property",
"schema_version": 1,
"name": "test foobar"
"account": "1969360",
"web_data": {
"conversion_rate": 1.49,
"average_order_value": 123,
"visitor_count": 25000,
"visits_count": 35000,
"revenue": 50000
}
}
There could be multiple documents with same account.
Now I should create MapReduce to calculate average of web_data.conversion_rate and web_data.average_order_value and sum of web_data.visitor_count, web_data.visits_count and web_data.revenue for each document which has same account.
Currently my mapping function emits array like this:
['10449266', 'test foobar']
where first item is account and second name.
Emitted value is just web_data.
I would like to have output like this:
GET /list_properties?group_level=1
{
rows: [
{
key: [
"10449266"
],
value: {
"property_count": 2,
"conversion_rate": 1.8625,
"average_order_value": 153.75,
"visitor_count": 31250,
"visits_count": 43750,
"revenue": 62500
}
},
{
key: [
"66294401"
],
value: { ... }
},
...
}
How could I achieve this?
Pretty complex pattern, huh?
I've tried so many reduce functions to do this, I'm totally exhausted now. This is one I tried to get started:
function (keys, values, rereduce) {
return values.reduce(function(a, b){
return a + b.web_data.revenue;
});
}
That doesn't work as it returns object instead of number. And I have no idea why. It sums up numbers and returns object?
EDIT:
I updated reduce function to this:
function (keys, values, rereduce) {
if(rereduce){
return values;
}else{
var revenue = values.reduce(function(a, b){
return a + b.web_data.revenue;
}, 0);
return {
revenue : revenue
};
}
}
Now I get response like this:
{
"key": [
"2035864"
],
"value": [
[
{
"revenue": 0
},
{
"revenue": 66321.2
},
{
"revenue": 5319.35
}
],
[
{
"revenue": 0
}
],
[
{
"revenue": 45432.02
}
],
[
{
"revenue": 185732.78
}
]
]
}
Now if I change return values to return values[0] in rereduce-part, I get response like:
{
"key": [
"2035864"
],
"value": {
"revenue": 0
}
}
NONSENSE AGAIN!
Everything I want is just:
{
"key": [
"2035864"
],
"value": {
"revenue": 302805.35
}
}
JavaScript allows to do this so easily in browser but I don't understand why CouchDB wants to do things as weirdly as this.
Goal:
I have a JSON payload with the following format:
{
"Values": [
{
"Details": {
"14342": {
"2016-06-07T00:00:00": {
"Value": 99.62,
"Count": 7186
},
"2016-06-08T00:00:00": {
"Value": 99.73,
"Count": 7492
}
},
"14362": {
"2016-06-07T00:00:00": {
"Value": 97.55,
"Count": 1879
},
"2016-06-08T00:00:00": {
"Value": 92.68,
"Count": 355
}
}
},
"Key": "query5570027",
"Total": 0.0
},
{
"Details": {
"14342": {
"2016-06-07T00:00:00": {
"Value": 0.0,
"Count": 1018
},
"2016-06-08T00:00:00": {
"Value": 0.0,
"Count": 1227
}
}
},
"Key": "query4004194",
"Total": 0.0
}
],
"LatencyInMinute": 0.0
}
I want to load this in PowerBI and produce a table like so:
Notice how each Value + Count pair has its own row and some elements are repeated.
Problem: When I try to do this in Power BI (via Power Query), I get three initial columns, one of which is Details. Trouble is that I can expand Details, but I just get more columns, where what I really want is rows. I tried transpose, pivoting columns, and such but nothing helped. My troubles are exacerbated by Power Query treating the nested data elements as column names.
Question: Is there a way, in M, to convert this nested JSON payload to the table example I illustrated above?
Chris Webb wrote a recursive function to expand all table-type columns - I've managed to clone it for record-type columns:
https://gist.github.com/Mike-Honey/0a252edf66c3c486b69b
If you use Record.FromList for the expansion it should work.
You can find an example in the script here: https://chris.koester.io/wp-content/uploads/2016/04/TransformJsonArrayWithPowerQueryImkeFeldmann.txt