I've been following the tutorial for batch operations:
https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-dynamodb-batch.html#multi-table-batch
When I tried to do a multi-table batch, I get a null response even though the objects are successfully submitted to dynamodb.
This is my mutation:
mutation sendReadings {
recordReadings(
tempReadings: [
{sensorId: 1, value: 85.5, timestamp: "2018-02-01T17:21:05.000+08:00"},
{sensorId: 2, value: 85.7, timestamp: "2018-02-01T17:21:06.000+08:00"},
{sensorId: 3, value: 85.8, timestamp: "2018-02-01T17:21:07.000+08:00"},
{sensorId: 4, value: 84.2, timestamp: "2018-02-01T17:21:08.000+08:00"},
{sensorId: 5, value: 81.5, timestamp: "2018-02-01T17:21:09.000+08:00"}
]
locReadings: [
{sensorId: 1, lat: 47.615063, long: -122.333551, timestamp: "2018-02-01T17:21:05.000+08:00"},
{sensorId: 2, lat: 47.615163, long: -122.333552, timestamp: "2018-02-01T17:21:06.000+08:00"}
{sensorId: 3, lat: 47.615263, long: -122.333553, timestamp: "2018-02-01T17:21:07.000+08:00"}
{sensorId: 4, lat: 47.615363, long: -122.333554, timestamp: "2018-02-01T17:21:08.000+08:00"}
{sensorId: 5, lat: 47.615463, long: -122.333555, timestamp: "2018-02-01T17:21:09.000+08:00"}
]) {
locationReadings {
sensorId
timestamp
lat
long
}
temperatureReadings {
sensorId
timestamp
value
}
}
}
The problem is that the response returns null:
{
"data": {
"recordReadings": {
"locationReadings": null,
"temperatureReadings": null
}
}
}
Request Mapping Template:
## Convert tempReadings arguments to DynamoDB objects
#set($tempReadings = [])
#foreach($reading in ${ctx.args.tempReadings})
$util.qr($tempReadings.add($util.dynamodb.toMapValues($reading)))
#end
## Convert locReadings arguments to DynamoDB objects
#set($locReadings = [])
#foreach($reading in ${ctx.args.locReadings})
$util.qr($locReadings.add($util.dynamodb.toMapValues($reading)))
#end
{
"version" : "2018-05-29",
"operation" : "BatchPutItem",
"tables" : {
"LocationReadingTable": $utils.toJson($locReadings),
"TemperatureReadingTable": $utils.toJson($tempReadings)
}
}
Response Mapping Template:
## If there was an error with the invocation
## there might have been partial results
#if($ctx.error)
## Append a GraphQL error for that field in the GraphQL response
$utils.appendError($ctx.error.message, $ctx.error.message)
#end
## Also returns data for the field in the GraphQL response
$utils.toJson($context.result.data)
The code is pretty much identical to the tutorial. I checked the logs and the response is being sent but it's not being captured so either my code is off or there is a bug on AWS's side. Can anyone reproduce this or is able to get a response when doing a multi-table BatchPutItem?
Thank you for a very detailed question, it certainly helped me debug your issue. So, in order to fix your issue, you need to tweak your response mapping template. I was able to reproduce the issue with null response on my end. The reason being that the shape of the multi-table batch put response is of this format:
"result": {
"data": {
"TemperatureReadingTable": [
{
"value": 85.5,
"sensorId": "1",
"timestamp": "2018-02-01T17:21:05.000+08:00"
},
{
"value": 85.7,
"sensorId": "2",
"timestamp": "2018-02-01T17:21:06.000+08:00"
},
{
"value": 85.8,
"sensorId": "3",
"timestamp": "2018-02-01T17:21:07.000+08:00"
},
{
"value": 84.2,
"sensorId": "4",
"timestamp": "2018-02-01T17:21:08.000+08:00"
},
{
"value": 81.5,
"sensorId": "5",
"timestamp": "2018-02-01T17:21:09.000+08:00"
}
],
"LocationReadingTable": [
{
"lat": 47.615063,
"long": -122.333551,
"sensorId": "1",
"timestamp": "2018-02-01T17:21:05.000+08:00"
},
{
"lat": 47.615163,
"long": -122.333552,
"sensorId": "2",
"timestamp": "2018-02-01T17:21:06.000+08:00"
},
{
"lat": 47.615263,
"long": -122.333553,
"sensorId": "3",
"timestamp": "2018-02-01T17:21:07.000+08:00"
},
{
"lat": 47.615363,
"long": -122.333554,
"sensorId": "4",
"timestamp": "2018-02-01T17:21:08.000+08:00"
},
{
"lat": 47.615463,
"long": -122.333555,
"sensorId": "5",
"timestamp": "2018-02-01T17:21:09.000+08:00"
}
]
},
So, the $context.result.data will return you an object with TemperatureReadingTable and LocationReadingTable as its fields. However, you are applying $util.toJson to resolve the type RecordResult, which has temperatureReadings and locationReadings as child fields.
Please update your Response Mapping template with the following definition, and it will work for you:
#if($ctx.error)
## Append a GraphQL error for that field in the GraphQL response
$utils.appendError($ctx.error.message, $ctx.error.message)
#end
{
"temperatureReadings": $util.toJson(${ctx.result.data.TemperatureReadingTable}),
"locationReadings": $util.toJson(${ctx.result.data.LocationReadingTable})
}
Moreover, I'd encourage you enable CloudWatch logs from the Settings page of the Console, with ALL as the option. This will log Request/Response headers, the resolved Request/Response templates, Tracing information, etc. So it'll help you identify issues like these quickly by looking at the content of the request/response templates.
Thank you for bringing this issue, we will update the documentation, if it does not mention this.
Related
Do we need to specify the hyperparameters both in algorithmHyperParameters and algorithmHyperParameterRanges? If yes, then should we specify a single value (string as per documentation) in algorithmHyperParameters, but a range (integer in case of integer-valued hyperparameter) in algorithmHyperParameterRanges?
For example: Similar-Items recipe has an integer-valued hyperparameter item_id_hidden_dimension. If I use the following solution_config, where item_id_hidden_dimension is specified only in algorithmHyperParameterRanges and not in algorithmHyperParameters, I get the error:
An error occurred (InvalidInputException) when calling the CreateSolution operation: Provide a hyperparameter that is used in the algorithm: arn:aws:personalize:::algorithm/aws-similar-items
"solution_config": {
"algorithmHyperParameters": {},
"hpoConfig": {
"algorithmHyperParameterRanges": {
"integerHyperParameterRanges": [
{
"name": "item_id_hidden_dimension",
"minValue": 30,
"maxValue": 200
}
],
"categoricalHyperParameterRanges": [],
"continuousHyperParameterRanges": []
},
"hpoResourceConfig": {
"maxNumberOfTrainingJobs": "4",
"maxParallelTrainingJobs": "2"
}
}
}
But if I use the following solution_config, where item_id_hidden_dimension is specified both in algorithmHyperParameterRanges and in algorithmHyperParameters, I still get the same error:
An error occurred (InvalidInputException) when calling the CreateSolution operation: Provide a hyperparameter that is used in the algorithm: arn:aws:personalize:::algorithm/aws-similar-items
"solution_config": {
"algorithmHyperParameters": {
"item_id_hidden_dimension": "100"
},
"hpoConfig": {
"algorithmHyperParameterRanges": {
"integerHyperParameterRanges": [
{
"name": "item_id_hidden_dimension",
"minValue": 30,
"maxValue": 200
}
],
"categoricalHyperParameterRanges": [],
"continuousHyperParameterRanges": []
},
"hpoResourceConfig": {
"maxNumberOfTrainingJobs": "4",
"maxParallelTrainingJobs": "2"
}
}
}
This is caused by an error in the documentation. The hyperparameter names should be item_id_hidden_dim and item_metadata_hidden_dim (note they are dim and not dimension as the documentation states).
This can be confirmed by calling the DescribeRecipe API to get the algorithmArn for the Similar-Items recipe and then calling the DescribeAlgorithm API to get details on the algorithm.
import boto3
import json
personalize = boto3.client('personalize')
response = personalize.describe_recipe(recipeArn = 'arn:aws:personalize:::recipe/aws-similar-items')
print(json.dumps(response['recipe'], indent=2, default=str))
{
"name": "aws-similar-items",
"recipeArn": "arn:aws:personalize:::recipe/aws-similar-items",
"algorithmArn": "arn:aws:personalize:::algorithm/aws-similar-items",
"featureTransformationArn": "arn:aws:personalize:::feature-transformation/similar-items",
"status": "ACTIVE",
"description": "Predicts items similar to a given item based on co-occurrence of items in the user-item interactions dataset and item metadata in the item dataset.",
"creationDateTime": "2019-06-10 00:00:00+00:00",
"recipeType": "RELATED_ITEMS",
"lastUpdatedDateTime": "2022-08-17 00:25:42.935000+00:00"
}
algo_arn = response['recipe']['algorithmArn']
response = personalize.describe_algorithm(algorithmArn = algo_arn)
print(json.dumps(response['algorithm'], indent=2, default=str))
{
"name": "aws-similar-items",
"algorithmArn": "arn:aws:personalize:::algorithm/aws-similar-items",
"algorithmImage": {
"name": "Item Similarity"
},
"defaultHyperParameters": {
"item_id_hidden_dim": "100",
"item_metadata_hidden_dim": "100"
},
"defaultHyperParameterRanges": {
"integerHyperParameterRanges": [
{
"name": "item_id_hidden_dim",
"minValue": 30,
"maxValue": 200,
"isTunable": true
},
{
"name": "item_metadata_hidden_dim",
"minValue": 30,
"maxValue": 200,
"isTunable": true
}
],
"continuousHyperParameterRanges": [],
"categoricalHyperParameterRanges": []
},
"defaultResourceConfig": {
"maxNumberOfTrainingJobs": "20",
"maxParallelTrainingJobs": "5"
},
"trainingInputMode": "File",
"creationDateTime": "2019-06-10 00:00:00+00:00",
"lastUpdatedDateTime": "2022-08-17 00:24:41.307000+00:00"
}
Note the hyperparameter names in the last response above.
We will get this error fixed in the documentation ASAP.
The problem is I can't perform PUT or POST with Postman app. But DELETE is possible. And yes I enabled all actions for public users.
Here is GET request result:
{
"data": [
{
"id": 2,
"attributes": {
"title": "23123",
"game": "1231",
"players": "2312313",
"createdAt": "2022-02-19T16:36:34.221Z",
"updatedAt": "2022-02-19T16:36:34.971Z"
}
}
],
"meta": {
"pagination": {
"page": 1,
"pageSize": 25,
"pageCount": 1,
"total": 1
}
}
}
And here is my entries for POST request:
finally after requesting POST here is result:
{
"data": null,
"error": {
"status": 400,
"name": "ValidationError",
"message": "Missing \"data\" payload in the request body",
"details": {}
}
}
What is the problem?
I finally found the solution. u need to put Body in raw and set it as Json! default is Text.
and POST example is :
{
"data": {
"title": "Hello",
"relation": 2,
"relations": [2, 4]
}
}
I want to use predict method in order to get recommendation devided by page.
This method has pageToken parameter used for paging. The parameter should be received in the previous prediction response.
But this method in v2 API doesn't return pageToken or nextPageToken that is in v1beta1 API.
Why it disappeared? How can I get next page for prediction?
Request url is
https://retail.googleapis.com/v2/projects/{PROJECT_NUMBER}/locations/global/catalogs/default_catalog/placements/recently_viewed_default:predict
Request body is
{
"filter": "tag=(\"成人作品\") filterOutOfStockItems",
"userEvent": {
"eventType": "home-page-view",
"visitorId": "1254704470.1607003607",
"productDetails": [
{
"product": {
"id": "220604"
},
"quantity": 1
}
]
}
}
The response is here. There is no nextPageToken
{
"results": [
{
"id": "170706"
},
{
"id": "64081"
},
:
(snip)
:
{
"id": "132940"
},
{
"id": "17557"
}
],
"attributionToken": "ChQxNzkzOTg1NDI3MTk1OTMyNzU0MhACGiNvZnRlbl9wdXJjX2ZyZXF1ZW50bHlfMTYxNDMwNjk3OTQyNSIYb2Z0ZW5fcHVyY2hhc2VkX3RvZ2V0aGVyKAA"
}
I am on halfway of implementing JSON API structure (with underscore attributes).
Actual state for development environment is:
I use the Active Model Adapter structure for requesting to the backend for resources and backend response me with JSON API structure.
In Application Serializer I am using JSONAPISerializer. I override methods:
serializeBelongsTo
keyForRelationship
keyForAttribute
serialize
serializeAttribute
serializeHasMany
and for development, everything works for me (backend in Rails communicate with Ember very good).
The problem is with Ember CLI Mirage and conventions (not sure if there are simple solutions or I need to override again methods in this addon).
Actual state with Ember Cli Mirage and test environment:
I am using import { JSONAPISerializer } from 'ember-cli-mirage';
and then trying to manipulate proper request and then transform it for JSON API format.
It could work like this:
Ember Adapter (Active Model Adapter format - with underscore attributes) ---> Mirage Serializer should get request (find resources created before in tests with associations) and then response it with JSON API format ---> JSON API Serializer could catch it and fill Ember DS.
For now, I have a missing part to serialize it for all cases to JSON API standard (with underscored attributes)
Where should I do this transformation to minimize overriding JSONAPISerializer Mirage Serializer.
I noticed that there are some helpers, but I have a problem to wrap this knowledge together
(http://www.ember-cli-mirage.com/docs/advanced/route-handlers#helpers)
UPDATE:
Example of structure from Backend:
{
"data": {
"id": "6",
"type": "first_resource",
"attributes": {
"id": 6,
"my_attribute": "my_attribute"
},
"relationships": {
"second_resources": {
"data": [
{
"id": "16",
"type": "second_resource"
}
]
},
"third_resource_other_type": {
"data": {
"id": "1",
"type": "third_resource"
}
},
"fourth_resource": {
"data": {
"id": "1",
"type": "fourth_resource"
}
}
},
"links": {
"fifth_resources": "/api/v1/first_resources/6/fifth_resources"
}
},
"included": [
{
"id": "1",
"type": "fourth_resource",
"attributes": {
"id": 1,
"my_attribute": "my_attribute"
},
"links": {
"sixth_resource": "/api/v1/fourth_resources/1/sixth_resource"
}
},
{
"id": "16",
"type": "second_resource",
"attributes": {
"id": 16,
"my_attribute": "my_attribute"
},
"relationships": {
"eighth_resources": {
"data": []
}
},
"links": {
"seventh_resources": "/api/v1/second_resources/16/seventh_resources"
}
},
{
"id": "17",
"type": "second_resource",
"attributes": {
"id": 17,
"my_attribute": "my_attribute"
},
"relationships": {
"eighth_resources": {
"data": []
}
},
"links": {
"seventh_resources": "/api/v1/second_resources/17/seventh_resources"
}
},
{
"id": "15",
"type": "second_resource",
"attributes": {
"id": 15,
"my_attribute": "my_attribute"
},
"relationships": {
"eighth_resources": {
"data": [
{
"id": "26",
"type": "eighth_resource"
},
{
"id": "24",
"type": "eighth_resource"
}
]
}
},
"links": {
"seventh_resources": "/api/v1/second_resources/15/seventh_resources"
}
},
{
"id": "26",
"type": "eighth_resource",
"attributes": {
"id": 26,
"my_attribute": "my_attribute"
}
}
]
}
UPDATE2
structure from mirage response:
data: {
attributes: {
my_attribute: 'my_attribute',
second_resource_ids: [36, 37],
fifth_resource_ids: []
},
id: 11,
relationships: {
third_resource_other_type: {data: null}
fourth_resource: {data: null}
second_resources: {data: []}
},
type: "first_resources"
}
resources in tests:
server.create('second-resource', {
id: 36,
first_resource_id: '11',
my_attribute: "my_attribute"
});
server.create('eighth-resource', {
id: 140,
second_resource_id: 37
});
server.create('eighth-resource', {
id: 141,
second_resource_id: 37
});
server.create('second-resource', {
id: 37,
first_resource_id: '11',
eighth_resource_ids: [140, 141]
});
server.create('first-resource', {
id: 11,
second_resource_ids: [36, 37]
});
first_resource model in mirage:
export default Model.extend({
third_resource_other_type: belongsTo(),
fourth_resource: belongsTo(),
fifth_resources: hasMany(),
second_resources: hasMany()
});
Let's try to focus a single relationship, since there's a lot going on in the question you've posted. We'll look at second-resource.
It looks like Mirage is sending back second_resource_ids under the attributes key of the first_resource primary data in the JSON:API payload. That tells me Mirage thinks second_resource_ids is an attribute of first_resource, when in fact it's a relationship.
Assuming your Models & Relationships are setup correctly, you need to tweak the way you're creating data in Mirage.
If you take a look at the Associations section of the Defining Routes guide, you'll see this message:
Mirage's database uses camelCase for all model attributes, including foreign keys (e.g. authorId in the example above)
Right now, you're doing this:
server.create('second-resource', {
id: 36,
first_resource_id: '11',
my_attribute: "my_attribute"
});
server.create('first-resource', {
id: 11,
second_resource_ids: [36, 37]
});
But from Mirage's perspective, you need to use camelCase IDs, or just pass in the relationships, to set these up correctly. Something like this:
let firstResource = server.create('first-resource', {
id: 11
});
server.create('second-resource', {
id: 36,
firstResource,
myAttribute: "my_attribute"
});
You could also pass in the foreign key on creation if you wanted - just be sure to use camelCase:
server.create('second-resource', {
id: 36,
firstResourceId: '11',
myAttribute: "my_attribute"
});
Just remember that the formatting decisions for things like attributes and foreign keys (things like some-attribute vs. some_attribute or relationship-id vs. relationship_id) are made at the serializer layer. When dealing with Mirage's ORM and database, you want to stick to camelCase, regardless of the format your Serializer emits.
For more information, have a look at these sections from the docs:
"Associations and serializers" section of the Quickstart
Defining relationships
"Working with relationships" section of the Factories guide
I have a couchdb record structure which looks like this
[
{
"app_version": 2,
"platform": "android",
"session": {
"timestamp": "2014-08-20T00:00:00.000Z",
"session_id": "TOnNIhCNQ31LlkpEPQ7XnN1D",
"ip": "202.150.213.66",
"location": "1.30324,103.5498"
}
},
{
"app_version": 2,
"platform": "android",
"session": {
"timestamp": "2014-08-21T00:00:00.000Z",
"session_id": "TOnNIhCNQ31LlkpEPQ7XnN1D",
"ip": "202.150.213.66",
"location": "1.30324,103.5498"
}
}
{
"app_version": 2,
"platform": "ios",
"session": {
"timestamp": "2014-08-21T00:00:00.000Z",
"session_id": "TOnNIhCNQ31LlkpEPQ7XnN1D",
"ip": "202.150.213.66",
"location": "1.30324,103.5498"
}
},
{
"app_version": 1,
"platform": "ios",
"session": {
"timestamp": "2014-08-21T00:00:00.000Z",
"session_id": "TOnNIhCNQ31LlkpEPQ7XnN1D",
"ip": "202.150.213.66",
"location": "1.30324,103.5498"
}
}
]
I need to query all the records which happened between a a given number of dates and a app_version number, and I want to get the total of each by the platform.
So I wrote a map-reduce function like this;
"total": {
"map": "function(doc) {
date = doc.session.timestamp.split("T")[0];
emit([date, doc.app_version,doc.platform], 1);
}",
"reduce": "_count"
}
This gives me the output properly by grouping the records into dates.
["2014-08-20", 2, "android"] 2
["2014-08-20", 2, "ios"] 1
["2014-08-21", 2, "android"] 1
["2014-08-21", 2, "ios"] 1
But the problem comes when I try to query them using the start_key and end_key (to query by the date range)
Im sending the GET request as follows;
http://localhost/dummy_db_new/_design/views/_view/total?
start_key=["2014-08-20",2,WHAT_TO_PUT_HERE]
&end_key=["2014-08-20",2,WHAT_TO_PUT_HERE]
&group=true
I need to know what to put at the above places for it to have any platform(a string).
Oh I was able to find an answer.
The Answer was to use a wildcard. So basically I sent the request with a wildcard which will accept any platform type
http://localhost/dummy_db_new/_design/views/_view/total?
start_key=["2014-08-20",2,0]
&end_key=["2014-08-20",2,{}]
&group=true
{} means javascript object, so it will accept any JS object.