I'm currently working on getting my data from getfeedback usabila to power BI. The idea is to not use any intermediate solution such as database or others, but to connect directly usabila API or webhook to power BI.
My problem is that the stream API on Power BI is configured to receive json between brackets [] like this:
[
{
"id" :"AAAAA555555"
}
]
and usabilla webhook sends JSON without the brackets []:
{
"id": "699aab85-9e27-45c1-9931-3e17fefbea46",
"client": {
"device": {
"type": "desktop",
"manufacturer": "Apple",
"product": "unknown"
},
"platform": {
"name": "Mac",
"version": {
"major": 10,
"minor": 15,
"patch": 7
}
}
I believe that we can't change neither the format in usabilla, nor on power BI. (But maybe I'm wrong and maybe there is another way to get data)..
Did someone already implemented this solution? How did you proceed?
Thank you so much,
Nora
I am currently working on recommendation AI. since I am new to GCP recommendation, I have been struggling with data format for catalog. I read the documentation and it says each product item JSON format should be on a single line.
I understand this totally, but It would be really great if I could get what the JSON format looks like in real because the one in their documentation is very ambiguous to me. and I am trying to use console to import data
I tried to import data looking like down below but I got error saying invalid JSON format 100 times. it has lots of reasons such as unexpected token and something should be there and so on.
[
{
"id": "1",
"title": "Toy Story (1995)",
"categories": [
"Animation",
"Children's",
"Comedy"
]
},
{
"id": "2",
"title": "Jumanji (1995)",
"categories": [
"Adventure",
"Children's",
"Fantasy"
]
},
...
]
Maybe it was because each item was not on a single line, but I am also wondering if the above is enough for importing. I am not sure if those data should be included in another property like
{
"inputConfig": {
"productInlineSource": {
"products": [
{
"id": "1",
"title": "Toy Story (1995)",
"categories": [
"Animation",
"Children's",
"Comedy"
]
},
{
"id": "2",
"title": "Jumanji (1995)",
"categories": [
"Adventure",
"Children's",
"Fantasy"
]
},
}
I can see the above in the documentation but it says it is for importing inline which is using POST request. it does not mention anything about importing with console. I just guess the format is also used for console but I am not 100% sure. that is why I am asking
Is there anyone who can show me the entire data format to import data by using console?
Problem Solved
For those who might have the same question, The exact data format you should import by using gcp console looks like
{"id":"1","title":"Toy Story (1995)","categories":["Animation","Children's","Comedy"]}
{"id":"2","title":"Jumanji (1995)","categories":["Adventure","Children's","Fantasy"]}
No square bracket wrapping all the items.
No comma between items.
Only each item on a single line.
Posting this Community Wiki for better visibility.
OP edited question and add solution:
The exact data format you should import by using gcp console looks like
{"id":"1","title":"Toy Story (1995)","categories":["Animation","Children's","Comedy"]}
{"id":"2","title":"Jumanji (1995)","categories":["Adventure","Children's","Fantasy"]}
No square bracket wrapping all the items.
No comma between items.
Only each item on a single line.
However I'd like to elaborate a bit.
There are a few ways to import Importing catalog information:
Importing catalog data from Merchant Center
Importing catalog data from BigQuery
Importing catalog data from Cloud Storage
I guess this is what was used by OP, as I was able to import catalog using UI and GCS with below JSON file.
{
"inputConfig": {
"catalogInlineSource": {
"catalogItems": [
{"id":"111","title":"Toy Story (1995)","categories":["Animation","Children's","Comedy"]}
{"id":"222","title":"Jumanji (1995)","categories":["Adventure","Children's","Fantasy"]}
{"id":"333","title":"Test Movie (2020)","categories":["Adventure","Children's","Fantasy"]}
]
}
}
}
Importing catalog data inline
At the bottom of the Importing catalog information documentation you can find information:
The line breaks are for readability; you should provide an entire catalog item on a single line. Each catalog item should be on its own line.
It means you should use something similar to NDJSON - convenient format for storing or streaming structured data that may be processed one record at a time.
If you would like to try inline method, you should use this format, however it's single line but with breaks for readability.
data.json file
{
"inputConfig": {
"catalogInlineSource": {
"catalogItems": [
{
"id": "1212",
"category_hierarchies": [ { "categories": [ "Animation", "Children's" ] } ],
"title": "Toy Story (1995)"
},
{
"id": "5858",
"category_hierarchies": [ { "categories": [ "Adventure", "Fantasy" ] } ],
"title": "Jumanji (1995)"
},
{
"id": "321123",
"category_hierarchies": [ { "categories": [ "Comedy", "Adventure" ] } ],
"title": "The Lord of the Rings: The Fellowship of the Ring (2001)"
},
]
}
}
}
Command
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
--data #./data.json \
"https://recommendationengine.googleapis.com/v1beta1/projects/[your-project]/locations/global/catalogs/default_catalog/catalogItems:import"
{
"name": "import-catalog-default_catalog-1179023525XX37366024",
"done": true
}
Please keep in mind that the above method requires Service Account authentication, otherwise you will get PERMISSION DENIED error.
"message" : "Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the translate.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.",
"status" : "PERMISSION_DENIED"
I am creating JSON file from Informatica. What I did so far:
configure the basic entity in Informatica MDM hub
create a Base object in the provisioning tool
Then, I accessed the predefined URL by Informatica like
"server:port"/cmx/cs/"databaseid"/"baseobjectname"/id.json.
By default, Informatica places the link attribute inside of the JSON file for parent/child/self if any
Is there any way we can remove the link attribute?
I am getting below output
{
"link": [
{
"href": "serveraddress//1.json?depth=2",
"rel": "children"
},
{
"href": "serveraddress//1.json",
"rel": "self"
}
],
"rowidObject": "2"
}
Expected:
{
"rowidObject": "2"
}
Finally i am able to solve this using the suppressLinks=true in the url call
http://:/cmx/cs//?q=''&depth=3&suppressLinks=true
I am trying to use Google Cloud Vision API.
I am using the REST API in this link.
POST https://vision.googleapis.com/v1/files:asyncBatchAnnotate
My request is
{
"requests": [
{
"inputConfig": {
"gcsSource": {
"uri": "gs://redaction-vision/pdf_page1_employment_request.pdf"
},
"mimeType": "application/pdf"
},
"features": [
{
"type": "DOCUMENT_TEXT_DETECTION"
}
],
"outputConfig": {
"gcsDestination": {
"uri": "gs://redaction-vision"
}
}
}
]
}
But the response is always only "name" like below:
{
"name": "operations/a7e4e40d1e1ac4c5"
}
My "gs" location is valid.
When I write the wrong path in "gcsSource", 404 not found error is coming.
Who knows why my response is weird?
This is expected, it will not send you the output as a HTTP response. To see what the API did, you need to go to your destination bucket and check for a file named "xxxxxxxxoutput-1-to-1.json", also, you need to specify the name of the object in your gcsDestination section, for example: gs://redaction-vision/test.
Since asyncBatchAnnotate is an asynchronous operation, it won't return the result, it instead returns the name of the operation. You can use that unique name to call GetOperation to check the status of the operation.
Note that there could be more than 1 output file for your pdf if the pdf has more pages than batchSize and the output json file names change depending on the number of pages. It isn't safe to always append "output-1-to-1.json".
Make sure that the uri prefix you put in the output config is unique because you have to do a wildcard search in gcs on the prefix you provide to get all of the json files that were created.
I have defined a restful service with nested queries. Output mapping is defined in XML. I get proper response as XML. But if I request a JSON response using Accept:Application/json I get
{
"Fault": {
"faultcode": "soapenv:Server",
"faultstring": "Error while writing to the output stream using JsonWriter",
"detail": ""
}
}
I was getting below exception in 3.5.0 and I found a jira saying it is fixed in 3.5.1. So I tried in 3.5.1 now I am not getting below exception but the same output.
javax.xml.stream.XMLStreamException: Invalid Staring element
Please note I have also tried the escapeNonPrintableChar="true" option in my queries but no use. Strange thing is it working for different data sets. Just one particular data set is throwing this output.
I have changed the JSON formatters as below and got it to work but there is a problem in that.
<messageFormatter contentType="application/json" class="org.apache.axis2.json.JSONMessageFormatter"/>
<!--messageFormatter contentType="application/json" class="org.apache.axis2.json.gson.JsonFormatter" / -->
<messageBuilder contentType="application/json" class="org.apache.axis2.json.JSONOMBuilder"/>
<!--messageBuilder contentType="application/json" class="org.apache.axis2.json.gson.JsonBuilder" /-->
If I use above formatter the null values are not represented properly. Like I get
"Person": {
"Name": {
"#nil": "true"
}
but I want it as (like the other JSON formatter used to give)
"Person": {
"Name": null
}
Any help please. Is there a bug still left in this area?
When you are creating the query in your output response, you define the format you want to receive the response you can select xml or json, in the case you mention you can select the json option, then select generate response this creates this json structure.
{
"entries": {
"entry": [
{
"field1": "$column1",
"field2": "$column2"
}
]
}
}
Then you can modify the answer you need with your fields. Here is an example of how I use it in my query
{
"Pharmacies": {
"Pharmacy": [
{
"ID": "$Id",
"Descripcion": "$Desc",
"Latitude": "$Latitude",
"Longitude": "$Longitude",
"Image": "$Image"
}
]
}
}
The values with "$" are correspond to the name of the column of the query
Regards