Thanks for your help,
I'm trying to transform the Array called 'Tags' hosting a list of pair of key-values into a list of columns: CustomerId, CustomerDisplayName, CustomerPath where each list contains a respective value being a store for each charge.
{
"periodFrom": "2020-11-09T00:00:00",
"periodTo": "2020-12-08T00:00:00",
"charges": [
{
"listPrice": 5.05,
"netPrice": 5.05,
"netPriceProrated": 5.05,
"subTotal": 5.05,
"currency": "CAD",
"isBilled": true,
"isProratable": true,
"deductions": [],
"fees": [],
"invoice": {
"number": "2822835",
"date": "2020-11-16T00:00:00",
"periodFrom": "2020-10-09T00:00:00",
"periodTo": "2020-11-08T00:00:00"
},
"taxes": [
{
"name": "GST",
"appliedRate": 5.0
},
{
"name": "QST",
"appliedRate": 9.975
}
],
"tags": [
{
"name": "CustomerId",
"value": "42c8edf4-365a-4068-bde6-33675832afbb"
},
{
"name": "CustomerDisplayName",
"value": "Blue Sky Group"
},
{
"name": "CustomerPath",
"value": "Devmesh Co-Branded/Blue Sky Group"
}
]
}
]
}
Here the actual snippet of code I'm actually
let
...
Response2 = Table.FromRecords( { Response } ),
#"Expand1" = Table.ExpandListColumn(Response2, "charges"),
#"Expand2" = Table.ExpandRecordColumn(#"Expand1", "charges", {"productId", "productName", "sku", "chargeId", "chargeName", "chargeType", "periodFrom", "periodTo", "quantity", "listPrice", "netPrice", "netPriceProrated", "subTotal", "currency", "isBilled", "isProratable", "deductions", "fees", "invoice", "taxes", "tags"}, {"charges.productId", "charges.productName", "charges.sku", "charges.chargeId", "charges.chargeName", "charges.chargeType", "charges.periodFrom", "charges.periodTo", "charges.quantity", "charges.listPrice", "charges.netPrice", "charges.netPriceProrated", "charges.subTotal", "charges.currency", "charges.isBilled", "charges.isProratable", "charges.deductions", "charges.fees", "charges.invoice", "charges.taxes", "charges.tags"})
in
#"Expand2"
Related
I have data with multiple dimensions, stored in the Druid cluster. for example, Data of movies and the revenue they earned from each country where they were screened.
I'm trying to build a query that the answer to be returned will be a table of all the movies, the total revenue of each of them, and the revenue for each country.
I succeeded to do it in Turnilo - it generated for me the following Druid query -
[
[
{
"queryType": "timeseries",
"dataSource": "movies_source",
"intervals": "2021-11-18T00:01Z/2021-11-21T00:01Z",
"granularity": "all",
"aggregations": [
{
"name": "__VALUE__",
"type": "doubleSum",
"fieldName": "revenue"
}
]
},
{
"queryType": "topN",
"dataSource": "movies_source",
"intervals": "2021-11-18T00:01Z/2021-11-21T00:01Z",
"granularity": "all",
"dimension": {
"type": "default",
"dimension": "movie_id",
"outputName": "movie_id"
},
"aggregations": [
{
"name": "revenue",
"type": "doubleSum",
"fieldName": "revenue"
}
],
"metric": "revenue",
"threshold": 50
}
],
[
{
"queryType": "topN",
"dataSource": "movies_source",
"intervals": "2021-11-18T00:01Z/2021-11-21T00:01Z",
"granularity": "all",
"filter": {
"type": "selector",
"dimension": "movie_id",
"value": "some_movie_id"
},
"dimension": {
"type": "default",
"dimension": "country",
"outputName": "country"
},
"aggregations": [
{
"name": "revenue",
"type": "doubleSum",
"fieldName": "revenue"
}
],
"metric": "revenue",
"threshold": 5
}
]
]
But it doesn't work when I'm trying to use it as a body for a Postman query - I got
{
"error": "Unknown exception",
"errorMessage": "Unexpected token (START_ARRAY), expected VALUE_STRING: need JSON String that contains type id (for subtype of org.apache.druid.query.Query)\n at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 2, column: 3]",
"errorClass": "com.fasterxml.jackson.databind.exc.MismatchedInputException",
"host": null
}
How should I build the corresponding query so that it works with Postman?
I am not familiar with Turnilo but have you tried using the Druid Console to write SQL and convert to Native request with the "Explain SQL query" option under the "Run/..." menu?
Your native queries seem to be doing a Top N instead of listing all movies, so I think the SQL might be something like:
SELECT movie_id, country_id, SUM(revenue) total_revenue
FROM movies_source
WHERE __time BETWEEN '2021-11-18 00:01:00' AND '2021-11-21 00:01:00'
GROUP BY movie_id, country_id
ORDER BY total_revenue DESC
LIMIT 50
I don't have the data source to test, but tested with sample wikipedia data with similar query structure:
SELECT namespace, cityName, sum(sum_added) total
FROM "wikipedia" r
WHERE cityName IS NOT NULL
AND __time BETWEEN '2015-09-12 00:00:00' AND '2015-09-15 00:00:00'
GROUP BY namespace, cityName
ORDER BY total DESC
limit 50
which results in the following Native query:
{
"queryType": "groupBy",
"dataSource": {
"type": "table",
"name": "wikipedia"
},
"intervals": {
"type": "intervals",
"intervals": [
"2015-09-12T00:00:00.000Z/2015-09-15T00:00:00.001Z"
]
},
"virtualColumns": [],
"filter": {
"type": "not",
"field": {
"type": "selector",
"dimension": "cityName",
"value": null,
"extractionFn": null
}
},
"granularity": {
"type": "all"
},
"dimensions": [
{
"type": "default",
"dimension": "namespace",
"outputName": "d0",
"outputType": "STRING"
},
{
"type": "default",
"dimension": "cityName",
"outputName": "d1",
"outputType": "STRING"
}
],
"aggregations": [
{
"type": "longSum",
"name": "a0",
"fieldName": "sum_added",
"expression": null
}
],
"postAggregations": [],
"having": null,
"limitSpec": {
"type": "default",
"columns": [
{
"dimension": "a0",
"direction": "descending",
"dimensionOrder": {
"type": "numeric"
}
}
],
"limit": 50
},
"context": {
"populateCache": false,
"sqlOuterLimit": 101,
"sqlQueryId": "cd5aabed-5e08-49b7-af63-fe82c125d3ee",
"useApproximateCountDistinct": false,
"useApproximateTopN": false,
"useCache": false
},
"descending": false
}
NOTE: this question is specifically for support staff of the HERE Developer API because they ask freemium users to post support questions on Stack Overflow rather than trying to contact them directly. If you're not a member of their staff and you're unable to help or if the question is unclear to you, don't worry about it. :)
For some reason the /discover endpoint doesn't return the "Buffalo Exchange" place that's at my specified coordinates, but only returns 2 localities that are much further away. This is the query that I'm using: https://discover.search.hereapi.com/v1/discover?at=34.003975%2C-118.484823&q=Buffalo&limit=20&apiKey=<insert API KEY>. These are the results I currently receive:
{
"items": [
{
"title": "Buffalo, NY, United States",
"id": "here:cm:namedplace:21018816",
"resultType": "locality",
"localityType": "city",
"address": {
"label": "Buffalo, NY, United States",
"countryCode": "USA",
"countryName": "United States",
"stateCode": "NY",
"state": "New York",
"county": "Erie",
"city": "Buffalo",
"postalCode": "14202"
},
"position": {
"lat": 42.88544,
"lng": -78.87846
},
"distance": 3551940,
"mapView": {
"west": -78.9168,
"south": 42.82603,
"east": -78.79492,
"north": 42.96651
}
},
{
"title": "Buffalo City, Eastern Cape, South Africa",
"id": "here:cm:namedplace:23402337",
"resultType": "locality",
"localityType": "city",
"address": {
"label": "Buffalo City, Eastern Cape, South Africa",
"countryCode": "ZAF",
"countryName": "South Africa",
"state": "Eastern Cape",
"county": "Buffalo City",
"city": "Buffalo City",
"postalCode": "5201"
},
"position": {
"lat": -33.0148,
"lng": 27.9038
},
"distance": 16910944,
"mapView": {
"west": 27.15745,
"south": -33.28749,
"east": 28.08053,
"north": -32.674
}
}
]
}
You can see that for both places the resultType is "locality".
Now compare that to the first result of a similar query that searches for the term "Exchange" instead of "Buffalo". All other query params are the same. This is the URL: https://discover.search.hereapi.com/v1/discover?at=34.003975%2C-118.484823&q=Exchange&limit=20&apiKey=<insert API KEY>, and this is how the results begin (not shown fully because there are many results):
{
"items": [
{
"title": "Buffalo Exchange",
"id": "here:pds:place:8403fv6k-b15f290ec4f409deea99318f7388bbd6",
"resultType": "place",
"address": {
"label": "Buffalo Exchange, 2449 Main St, Santa Monica, CA 90405, United States",
"countryCode": "USA",
"countryName": "United States",
"stateCode": "CA",
"state": "California",
"county": "Los Angeles",
"city": "Santa Monica",
"district": "City of Santa Monica",
"street": "Main St",
"postalCode": "90405",
"houseNumber": "2449"
},
"position": {
"lat": 34.00342,
"lng": -118.48483
},
"access": [
{
"lat": 34.00331,
"lng": -118.48493
}
],
"distance": 61,
"categories": [
{
"id": "600-6800-0090",
"name": "Women's Apparel",
"primary": true
},
{
"id": "600-6800-0000",
"name": "Clothing & Accessories"
},
{
"id": "600-6800-0089",
"name": "Men's Apparel"
},
{
"id": "600-6900-0251",
"name": "Used/Second-hand Merchandise Stores"
}
],
"references": [
{
"supplier": {
"id": "core"
},
"id": "1211447153"
},
{
"supplier": {
"id": "yelp"
},
"id": "5PzeN6hGLBPmJpCJ2ZmfCQ"
}
],
"contacts": [
{
"phone": [
{
"value": "+13103147300"
},
{
"value": "+13103924301",
"categories": [
{
"id": "600-6800-0000"
}
]
}
],
"fax": [
{
"value": "(520) 622-7015",
"categories": [
{
"id": "600-6800-0000"
}
]
}
],
"www": [
{
"value": "http://www.buffaloexchange.com",
"categories": [
{
"id": "600-6800-0000"
},
{
"id": "600-6900-0251"
}
]
}
],
"email": [
{
"value": "contact#bufex.com",
"categories": [
{
"id": "600-6800-0000"
}
]
}
]
}
],
"openingHours": [
{
"categories": [
{
"id": "600-6800-0000"
}
],
"text": [
"Mon-Sun: 11:00 - 20:00"
],
"isOpen": false,
"structured": [
{
"start": "T110000",
"duration": "PT09H00M",
"recurrence": "FREQ:DAILY;BYDAY:MO,TU,WE,TH,FR,SA,SU"
}
]
},
{
"categories": [
{
"id": "600-6800-0090"
},
{
"id": "600-6900-0251"
}
],
"text": [
"Mon-Sat: 11:00 - 20:00",
"Sun: 11:00 - 19:00"
],
"isOpen": false,
"structured": [
{
"start": "T110000",
"duration": "PT09H00M",
"recurrence": "FREQ:DAILY;BYDAY:MO,TU,WE,TH,FR,SA"
},
{
"start": "T110000",
"duration": "PT08H00M",
"recurrence": "FREQ:DAILY;BYDAY:SU"
}
]
}
]
},
...
}
You can see that the first result has the name "Buffalo Exchange" and the "resultType" is "place". This is the result I want. The question is why does this result fail to show up when the search query is "Buffalo"? Of course with the /discover endpoint I can't specify the category IDs I want to search, that's only available via the /browse endpoint. But with the /browse endpoint I can't specify a specific search term like "Buffalo" or "Exchange".
Update: this problem also happens with the "Bison" query in Alberta, Canada. The query for this is https://discover.search.hereapi.com/v1/discover?at=56.745531%2C-111.351341&q=Exchange&limit=20&apiKey=<insert API KEY>. This query yields only 10 results, and only 4 of them have resultType of "place".
We recommend application developers to use both Autosuggest and Discover, you get the nearby Buffalo Exchange because Autosuggest considers that the query is incomplete. Note that Discover considers the query to be complete.
As HERE Geocoding and Search is meant to provide relevant responses to user queries.
Geocoding Search Api (Autosuggest): https://developer.here.com/documentation/geocoding-search-api/dev_guide/topics/endpoint-autosuggest-brief.html
Geocoding Search Api (Discover): https://developer.here.com/documentation/geocoding-search-api/dev_guide/topics/endpoint-discover-brief.html
The above-mentioned user guide can help modify the query for accurate results.
For given a dictionary of list of dictionary below, I would like to check if key value "url": "/Province/?ID=83" exist in a list of dictionary or not.
data = {
"current_page": 1,
"data": [
{
"columns": [
{
"key": "ID",
"value": "83"
},
{
"key": "Description",
"value": "Test Curring"
},
{
"key": "LocalDescription",
"value": "tests curring"
}
],
"important": [
{
"key": "ID",
"value": "83"
},
{
"key": "Description",
"value": "Test Curring"
},
{
"key": "LocalDescription",
"value": "tests curring"
}
],
"url": "/Province/?ID=83"
},
{
"columns": [
{
"key": "ID",
"value": "82"
},
{
"key": "Description",
"value": "Test 81 Description"
},
{
"key": "LocalDescription",
"value": "Local Test Description"
}
],
"important": [
{
"key": "ID",
"value": "82"
},
{
"key": "Description",
"value": "Test 81 Description"
},
{
"key": "LocalDescription",
"value": "Local Test Description"
}
],
"url": "/Province/?ID=82"
},
]
}
I know it exist, how can I check to return True or False for its existence?
I tried:
list_data = data.get('data')
index = 0
for item in list_data:
url = item[0].get('url')
index += 1
print 'url: ',url
However, I got error:
url = item[0].get('url')
KeyError: 0
This works by gathering url of each item in data['data'] and seeing if the desired value is in it; it assumes every item in data['data'] has a url element.
"/Province/?ID=83" in [x['url'] for x in data['data']]
Assuming that the schema of data is consistent, you can try this:
It will return True if key:value exist in data, else it will return False.
def check(data, key, value):
for i in data['data']:
try:
if(i[key]==value): return True
except:
pass
return False
I have a table for some activities like
[
{
"id": 123,
"name": "Ram",
"status": 1,
"activity": "Poster Design"
},
{
"id": 123,
"name": "Ram",
"status": 1,
"activity": "Poster Design"
},
{
"id": 124,
"name": "Leo",
"categories": [
"A",
"B",
"C"
],
"status": 1,
"activity": "Brochure"
},
{
"id": 134,
"name": "Levin",
"categories": [
"A",
"B",
"C"
],
"status": 1,
"activity": "3D Printing"
}
]
I want to get this data from elastic search 5.5 by sorting on field activity, but I need all the data corresponding to name = "Ram" first and then remaining in a single query.
You can use function score query to boost the result based on match for the filter(this case ram in name).
Following query should work for you
POST sort_index/_search
{
"query": {
"function_score": {
"query": {
"match_all": {}
},
"boost": "5",
"functions": [{
"filter": {
"match": {
"name": "ram"
}
},
"random_score": {},
"weight": 1000
}],
"score_mode": "max"
}
},
"sort": [{
"activity.keyword": {
"order": "desc"
}
}]
}
I would suggest using a bool query combined with the should clause.
U will also need to use the sort clause on your field.
I am new to CouchDB. I have a 9 gb dataset loaded into my couchdb. I am able to map everything correctly. But I cannot reduce any of the results using the code written in the reduce column. When i tried log, log shows that rereduce values as false. Do i need to do anything special while doing the Map() or how to set the rereduce value is TRUE??
A sample of my data is as follows:
{
"_id": "33d4d945613344f13a3ee92933b160bf",
"_rev": "1-0425ca93e3aa939dff46dd51c3ab86f2",
"release": {
"genres": {
"genre": "Electronic"
},
"status": "Accepted",
"videos": {
"video": [
{
"title": "[1995] bola - krak jakomo",
"duration": 349,
"description": "[1995] bola - krak jakomo",
"src": "http://www.youtube.com/watch?v=KrELXoYThpI",
"embed": true
},
{
"title": "Bola - Forcasa 3",
"duration": 325,
"description": "Bola - Forcasa 3",
"src": "http://www.youtube.com/watch?v=Lz9itUo5xtc",
"embed": true
},
{
"title": "Bola (Darrell Fitton) - Metalurg (MV)",
"duration": 439,
"description": "Bola (Darrell Fitton) - Metalurg (MV)",
"src": "http://www.youtube.com/watch?v=_MYpOOMRAeQ",
"embed": true
}
]
},
"labels": {
"label": {
"catno": "SKA005",
"name": "Skam"
}
},
"companies": "",
"styles": {
"style": [
"Downtempo",
"Experimental",
"Ambient"
]
},
"formats": {
"format": {
"text": "",
"name": "Vinyl",
"qty": 1,
"descriptions": {
"description": [
"12\"",
"Limited Edition",
"33 ⅓ RPM"
]
}
}
},
"country": "UK",
"id": 1928,
"released": "1995-00-00",
"artists": {
"artist": {
"id": 390,
"anv": "",
"name": "Bola",
"role": "",
"tracks": "",
"join": ""
}
},
"title": 1,
"master_id": 13562,
"tracklist": {
"track": [
{
"position": "A1",
"duration": "4:33",
"title": "Forcasa 3"
},
{
"position": "A2",
"duration": "5:48",
"title": "Krak Jakomo"
},
{
"position": "B1",
"duration": "7:50",
"title": "Metalurg 2"
},
{
"position": "B2",
"duration": "6:40",
"title": "Balloom"
}
]
},
"data_quality": "Correct",
"extraartists": {
"artist": {
"id": 388200,
"anv": "",
"name": "Paul Solomons",
"role": "Mastered By",
"tracks": "",
"join": ""
}
},
"notes": "Limited to 480 copies.\nA1 is a shorter version than that found on the 'Soup' LP.\nA2 ends in a lock groove."
}
}
My intention is to count the mapped values. My mapping function is as follows:
function(doc){
if(doc.release)
emit(doc.release.title,1)
}
Map results shows around 5800 results
I want to use the following functions in the reduce tab to count:
Reduce:
_count or _sum
It does not give single rounded value. Even i cannot get the simple _count operations right !!! :(
for screenshot,
Please help me !!!
What you got was the sum of values per title. What you wanted, was the sum of values in general.
Change the grouping drop-down list to none.
Check CouchdDB's wiki for more details on grouping.