Cloud API Vision Results not appearing - google-cloud-platform

I'm making a request with the google vision api that appears to have worked, I get an operation number back. The problem I am having is the I am not sure how to interpret the results and nothing appeared in the output folder after running the script.
This is the script I ran
https://vision.googleapis.com/v1/files:asyncBatchAnnotate
{
"requests":[
{
"inputConfig": {
"gcsSource": {
"uri": "gs://somebucket/1.pdf"
},
"mimeType": "application/pdf"
},
"features": [
{
"type": "DOCUMENT_TEXT_DETECTION"
}
],
"outputConfig": {
"gcsDestination": {
"uri": "gs://somebucket/output/"
},
"batchSize": 1
}
}
]
}
This returns back
{
"name": "operations/8b7534d4b21b825e"
}
and when I do a lookup on the operation I get this
https://vision.googleapis.com/v1/operations/8b7534d4b21b825e
{
"name": "operations/8b7534d4b21b825e",
"metadata": {
"#type": "type.googleapis.com/google.cloud.vision.v1.OperationMetadata",
"state": "CREATED",
"createTime": "2019-01-09T21:08:57.339363096Z",
"updateTime": "2019-01-09T21:08:57.339363096Z"
}
}
However the output folder is completely empty and I am not sure what to make of the state created.

According to this answer by a Google engineer, latency in the order of minutes (~10 minutes) is somewhat expected. I’ve done some tests myself, with small files and at moments delay can be up to 25 minutes, though in some cases it is much less.
When Vision API is done processing your request, you should get a response like the one below, for the get method:
{
"name": "operations/XXXxxxxXXXX",
"metadata": {
"#type": "type.googleapis.com/google.cloud.vision.v1.OperationMetadata",
"state": "DONE",
"createTime": "2019-01-09T23:08:37.312889645Z",
"updateTime": "2019-01-09T23:08:59.169306747Z"
},
"done": true,
"response": {
"#type": "type.googleapis.com/google.cloud.vision.v1.AsyncBatchAnnotateFilesResponse",
"responses": [
{
"outputConfig": {
"gcsDestination": {
"uri": "gs://somebucket/output/"
}
}
}
]
}
}

Related

Monitoring api in Google gives "By" as response

I am reading monitoring data through Google Timeseries api. The api is working correctly and if give alignment period=3600s it gives me the values for that time series between start and end time for any metric type.
I am calling it through Python like this:
service.projects().timeSeries().list(
name=api_args["project_name"],
filter=api_args["metric_filter"],
aggregation_alignmentPeriod=api_args["aggregation_alignment_period"],
# aggregation_crossSeriesReducer=api_args["crossSeriesReducer"],
aggregation_perSeriesAligner=api_args["perSeriesAligner"],
aggregation_groupByFields=api_args["group_by"],
interval_endTime=api_args["end_time_str"],
interval_startTime=api_args["start_time_str"],
pageSize=config.PAGE_SIZE,
pageToken=api_args["nextPageToken"]
).execute()
and in Postman:
https://monitoring.googleapis.com/v3/projects/my-project/timeSeries?pageSize=500&interval.startTime=2020-07-04T16%3A39%3A37.230000Z&aggregation.alignmentPeriod=3600s&aggregation.perSeriesAligner=ALIGN_SUM&filter=metric.type%3D%22compute.googleapis.com%2Finstance%2Fnetwork%2Freceived_bytes_count%22+&pageToken=&interval.endTime=2020-07-04T17%3A30%3A01.497Z&alt=json&aggregation.groupByFields=metric.labels.key
I face an issue here:
{
"metric": {
"labels": {
"instance_name": "insta-demo1",
"loadbalanced": "false"
},
"type": "compute.googleapis.com/instance/network/received_bytes_count"
},
"resource": {
"type": "gce_instance",
"labels": {
"instance_id": "1234343552",
"zone": "us-central1-f",
"project_id": "my-project"
}
},
"metricKind": "DELTA",
"valueType": "INT64",
"points": [
{
"interval": {
"startTime": "2020-07-04T16:30:01.497Z",
"endTime": "2020-07-04T17:30:01.497Z"
},
"value": {
"int64Value": "6720271"
}
}
]
},
{
"metric": {
"labels": {
"loadbalanced": "true",
"instance_name": "insta-demo2"
},
"type": "compute.googleapis.com/instance/network/received_bytes_count"
},
"resource": {
"type": "gce_instance",
"labels": {
"instance_id": "1234566343",
"project_id": "my-project",
"zone": "us-central1-f"
}
},
"metricKind": "DELTA",
"valueType": "INT64",
"points": [
{
"interval": {
"startTime": "2020-07-04T16:30:01.497Z",
"endTime": "2020-07-04T17:30:01.497Z"
},
"value": {
"int64Value": "579187"
}
}
]
}
],
"unit": "By". //This "By" is the value which is causing problem,
I am getting this value like "unit": "By" or "unit":"ms" or something like that at the end, Also if I don't find any data for a range I'm getting this value, as I am evaluating this response in Python I am getting key error as there is not key called "unit"
logMessage: "Key Error: ' '"
severity: "ERROR"
As the response is empty I am getting the single key called "unit". Also at the end of any response I am getting this "unit":"ms" or "unit":"by" - is there any way to prevent that unit value coming in the response?
I am new to Google Cloud APIs and Python. What can I try next?
The "unit" field expresses the kind of resource the metric is counting. For bytes, it is "By". Read this. I understand it is always returned, so there is no way of not receiving it; I recommend you to adapt your code to correctly deal with its appearance in the responses.

How to skip particular set of test cases in a collection in terminal using newman?

I am having a postman collection which consists of request and test cases for each requests. I have two test case for each request. one for validating status code and other for validating response time. I need to execute status code test case frequently and response time test case results occasionally.How to achieve it without modifying the collection for every run and is it achievable in providing any option in terminal?
collection.json
{
"name": "Metadata",
"item": [
{
"name": "info",
"event": [
{
"listen": "test",
"script": {
"id": "32cf67e7-5d42-4231-86fe-e7fffa32c855",
"exec": [
"pm.test(\"Status code is 200\", function () {",
" pm.response.to.have.status(200);",
"});",
"pm.test(\"Response time is less than 300ms\", function () {",
" pm.expect(pm.response.responseTime).to.be.below(300);",
"});"
],
"type": "text/javascript"
}
}
],
"request": {
"auth": {
"type": "bearer",
"bearer": [
{
"key": "token",
"value": "{{tokenAdmin}}",
"type": "string"
}
]
},
"method": "GET",
"header": [],
"url": {
"raw": "{{url}}/api/m0/metadata/info",
"host": [
"{{url}}"
],
"path": [
"api",
"m0",
"metadata",
"info"
]
}
},
"response": []
}
],
"protocolProfileBehavior": {},
"_postman_isSubFolder": true
}
For a very basic flow, you can use moment to check which day it currently is and if that matches the condition, it will run the responseTime test.
let moment = require('moment'),
date = moment().format('dddd');
// Runs on each request
pm.test("Status code is 200", function () {
pm.response.to.have.status(200);
});
// Only runs on a Friday
if (date === 'Friday') {
pm.test("Response time is less than 1000ms", function () {
pm.expect(pm.response.responseTime).to.be.below(1000);
});
}
Moment has lots of different options available to you and might work if you want to only run that check at the end of the sprint or on a given day.

Alexa SmartHome Skills : Issue with device discovery

I have written an Alexa smart home skills.
When I try to discover the device using the Alexa test or from the mobile app, the lambda is triggered.
The lambda is getting successfully executed, but I get below error in App or test in Alexa console.
I couldn't find any new Smart Home devices. If you’ve ‎n't already,
please enable the smart home skill for your device from the Alexa App.
What could be the possible issue?
Since the lambda is getting successfully executed, I don't think there is any issue with language (English(IN)) or AWS region (EU-WEST-1) , where the lambda is deployed.
I didn't see any logs on Alexa developer console
Any pointers?
Response from Lambda function -
header =
{
namespace: 'Alexa.Discovery',
name: 'Discover.Response',
payloadVersion: '3',
messageId: '785f0173-6ddb-41d8-a785-de7159c7f7ca'
}
payload =
{
"endpoints": [
{
"endpointId": "d4b87cbe6c8e490493733f260b8c2c25",
"friendlyName": "Kitchen",
"description": "Demo",
"manufacturerName": "Man1",
"displayCategories": [
"LIGHT"
],
"cookie": {
"owner": "Owner1"
},
"capabilities": [
{
"type": "AlexaInterface",
"version": "3",
"interface": "Alexa"
},
{
"type": "AlexaInterface",
"version": "3",
"interface": "Alexa.PowerController",
"properties": {
"supported": [
{
"name": "powerState"
}
],
"proactivelyReported": true,
"retrievable": true
}
},
{
"type": "AlexaInterface",
"version": "3",
"interface": "Alexa.BrightnessController",
"properties": {
"supported": [
{
"name": "brightness"
}
],
"proactivelyReported": true,
"retrievable": true
}
}
]
}
]
}
We are wrapping header and payload in the response event.
context.succeed({ event: { header: header, payload: payload } });
So far I haven't found a way to view the logs either.
I had the same problem and I realized that I was putting wrong values in some properties or schema entities like ids.
In the same way, another thing that solved me on some occasion was to place the scheme in the following way:
context.succeed({
"event": {
"header": {
"namespace": "Alexa.Discovery",
"name": "Discover.Response",
"payloadVersion": "3",
"messageId": header.messageId
},
"payload": {
"endpoints": [
{
"endpointId": "demo_id",
...
,
"cookie": {},
"capabilities": [
{
"type": "AlexaInterface",
"interface": "Alexa",
"version": "3"
},
...
]
}
]
}
}
});

Cryptic error from request that had been working

I wanted to know if there were any changes to the dlp api in regards to authentication or whether the service may be down. Seems like the docs don't mention any changes.
The request is formated as such
POST https://dlp.googleapis.com/v2/projects/<project_id>/content:deidentify?key=<key> HTTP/1.1
{
"item": {
"value": "[\"Humphrey\",\"Roy\",\"Hodge\",\"Juarez\",\"Watkins\",\"Calderon\",\"Mayer\",\"Drake\",\"Valdez\",\"Landry\"]"
},
"deidentifyConfig": {
"infoTypeTransformations": {
"transformations": [
{
"primitiveTransformation": {
"replaceWithInfoTypeConfig": {}
}
}
]
}
},
"inspectConfig": {
"infoTypes": [
{
"name": "US_SOCIAL_SECURITY_NUMBER"
},
{
"name": "CREDIT_CARD_NUMBER"
},
{
"name": "US_DRIVERS_LICENSE_NUMBER"
},
{
"name": "EMAIL_ADDRESS"
},
{
"name": "PERSON_NAME"
},
{
"name": "PHONE_NUMBER"
},
{
"name": "LOCATION"
}
]
}
}
The error returned is very cryptic and thus I'm having a hard time figuring out what exactly is going on especially given that the exact same request had been working earlier.
{
"error": {
"code": 500,
"message": "Internal error encountered.",
"status": "INTERNAL"
}
}
There is indeed a bug, the engineering team has a patch for it and is in the process of rolling out the fix. (it only impacts calls unauthenticated, so calling with auth is the work around)

How do you handle large relationship data attributes and compound documents?

If an article has several comments (think thousands over time). Should data.relationships.comments return with a limit?
{
"data": [
{
"type": "articles",
"id": 1,
"attributes": {
"title": "Some title",
},
"relationships": {
"comments": {
"links": {
"related": "https://www.foo.com/api/v1/articles/1/comments"
},
"data": [
{ "type": "comment", "id": "1" }
...
{ "type": "comment", "id": "2000" }
]
}
}
}
],
"included": [
{
"type": "comments",
"id": 1,
"attributes": {
"body": "Lorem ipusm",
}
},
.....
{
"type": "comments",
"id": 2000,
"attributes": {
"body": "Lorem ipusm",
}
},
]
}
This starts to feel concerning, when you think of compound documents (http://jsonapi.org/format/#document-compound-documents). Which means, the included section will list all comments as well, making the JSON payload quite large.
If you want to limit the number of records you get at a time from a long list use pagination (JSON API spec).
I would load the comments separately with store.query (ember docs), like so -
store.query('comments', { author_id: <author_id>, page: 3 });
which will return the relevant subset of comments.
If you don't initially want to make two requests per author, you could include the first 'page' in the authors request as you're doing now.
You may also want to look into an addon like Ember Infinity (untested), which will provide an infinite scrolling list and automatically make pagination requests.