Struggling with this for sometime now, and applogies I changed the query name for the question to getDeviceReadings, I have been using getAllUserDevices (sorry for any confusion)
type Device {
id: String
device: String!
}
type Reading {
device: String
time: Int
}
type PaginatedDevices {
devices: [Device]
readings: [Reading]
nextToken: String
}
type Query {
getDevicesReadings(nextToken: String, count: Int): PaginatedDevices
}
Then I have a resolver on the query getDevicesReadings which works fine and returns all the devices a user has so far so good
{
"version": "2017-02-28",
"operation": "Query",
"query" : {
"expression": "id = :id",
"expressionValues" : {
":id" : { "S" : "${context.identity.username}" }
}
}
#if( ${context.arguments.count} )
,"limit": ${context.arguments.count}
#end
#if( ${context.arguments.nextToken} )
,"nextToken": "${context.arguments.nextToken}"
#end
}
now I want to return all the readings that devices has based on the source result so I have a resolver on getDevicesReadings/readings
#set($ids = [])
#foreach($id in ${ctx.source.devices})
#set($map = {})
$util.qr($map.put("device", $util.dynamodb.toString($id.device)))
$util.qr($ids.add($map))
#end
{
"version" : "2018-05-29",
"operation" : "BatchGetItem",
"tables" : {
"readings": {
"keys": $util.toJson($ids),
"consistentRead": true
}
}
}
With a response mapping like so ..
$utils.toJson($context.result.data.readings)
I run a query
query getShit{
getDevicesReadings{
devices{
device
}
readings{
device
time
}
}
}
this returns the following results
{
"data": {
"getAllUserDevices": {
"devices": [
{
"device": "123"
},
{
"device": "a935eeb8-a0d0-11e8-a020-7c67a28eda41"
}
],
"readings": [
null,
null
]
}
}
}
As you can see on the image the primary partition key is device on the readings table I look at the logs and I have the following
Sorry if you cant read the log it basically says that there are unprocessedKeys
and the following error message
"message": "The provided key element does not match the schema (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: 0H21LJE234CH1GO7A705VNQTJVVV4KQNSO5AEMVJF66Q9ASUAAJG)",
I'm guessing some how my mapping isn't quite correct and I'm passing in readings as my keys ?
Any help greatly appreciated
No, you can absolutely use batch resolvers when you have a primary sort key. The error in your example is that you were not providing the primary sort key to the resolver.
This code needs to provide a "time" as well a "device" because you need both to fully specify the primary key.
#set($ids = [])
#foreach($id in ${ctx.source.devices})
#set($map = {})
$util.qr($map.put("device", $util.dynamodb.toString($id.device)))
$util.qr($ids.add($map))
#end
You should have something like it:
#set($ids = [])
#foreach($id in ${ctx.source.devices})
#set($map = {})
# The tables primary key is made up of "device" AND "time"
$util.qr($map.put("device", $util.dynamodb.toString($id.device)))
$util.qr($map.put("time", $util.dynamodb.toString($id.time)))
$util.qr($ids.add($map))
#end
If you want to get many records that share the same "device" value but that have different "time" values, you need to use a DynamoDB Query operation, not a batch get.
You're correct, the request mapping template you provided doesn't match the primary key on the readings table. A BatchGetItem expects keys to be primary keys, however you are only passing the hash key.
For the BatchGetItem call to succeed you must pass both hash and sort key, so in this case, both device and time attributes.
Maybe a Query on the readings table would be more appropriate?
So you can't have a batch resolver when you have primary sort key ?!
So the answer was to create a lambda function and tack that on as my resolver
import boto3
from boto3.dynamodb.conditions import Key
def lambda_handler(event, context):
list = []
for device in event['source']['devices'] :
dynamodb = boto3.resource('dynamodb')
readings = dynamodb.Table('readings')
response = readings.query(
KeyConditionExpression=Key('device').eq(device['device'])
)
items = response['Items']
list.extend(items)
return list
Related
I want to write a query to match indexed fields in Elasticsearch. I am using AWS Elasticsearch service and writing the query as an AWS Lambda function. This lambda function is executed when an event occurs, searches for a fields sent in the event, matches the fields with the indexed documents and returns the matched documents.
However, we don't know the fields or the number of fields to be searched ahead of time. So I want to be able to extract the fields from the event in the lambda function and construct the query dynamically to match the fields with the indexed documents.
The event is as follows:
{
"headers": {
"Host": "***"
},
"queryStringParameters": {
"fieldA": "abc",
"fieldB": "def"
}
}
The lambda function is as follows. This function expects two fields and matches them.
def search(event, context):
fields = list(event['queryStringParameters'].keys())
firstField = fields[0]
secondField = fields[1]
values = list(event['queryStringParameters'].values())
firstValue = values[0]
secondValue = values[1]
query = {
"query": {
"bool" : {
"must" :[
{"match" : { firstField : firstValue }},
{"match" : { secondField : secondValue }}
]
}
}
}
How can I rewrite my query so it dynamically accepts the fields and the number of fields that the event sends (not known ahead of time)?
Not sure what your exact requirements are but you could go with the following:
def search(event, context):
query = {
"query": {
"bool": {
"query_string": {
"query": " OR ".join([
"(%s:'%s')" % (k, v) for (k, v) in event["queryStringParameters"].items()
])
}
}
}
}
print(query)
which'd result in a proper query_string_query:
{
"query":{
"bool":{
"query_string":{
"query":"(fieldB:'def') OR (fieldA:'abc')"
}
}
}
}
You could interchange the OR with an AND. Also keep in mind that when the values are wrapped in quotes, ES will enforce exact matches. Leave them out in case you're after a contains behavior (i.e. a match query).
This is my first time working with VTL, so please correct my code if there is something very silly in it.
What I want to achieve
{
"cities": ["jaipur", "mumbai", "delhi", "sheros", "jalandhar", "bengaluru"]
}
Graphql schema:
type Query{
getCitiesForGroups: String
}
Default response template:
#if($ctx.error)
$utils.error($ctx.error.message, $ctx.error.type)
#end
$utils.toJson($utils.rds.toJsonObject($ctx.result)[0])
Using the default response template, the result I was getting
{ "data": { "getCitiesForGroups": [ "{groupcity=jaipur}", "{groupcity=mumbai}", "{groupcity=delhi}", "{groupcity=sheros}", "{groupcity=jalandhar}", "{groupcity=bengaluru}" ] } }
My request template
{
"version": "2018-05-29",
"statements": [
"select DISTINCT LOWER(city) as city from public.Groups"
]
}
To get the desired output, I changed the response template as I wanted to loop over the response I got from the DB, and remove the {city=} from the string by using the substring method given in AWS resolver mapping docs and this is where I am facing the problem.
My response template
#if($ctx.error)
$utils.error($ctx.error.message, $ctx.error.type)
#end
#set ($rawListOfCities = $utils.rds.toJsonObject($ctx.result)[0])
#set ($sanitisedListOfCities = [])
#foreach( $city in $rawListOfCities )
#set ($equalToIndex = $city.indexOf("="))
#set ($equalToIndex = $equalToIndex + 1)
#set ($curlyLastIndex = $city.lastIndexOf("}"))
#set ($tempCity = $city.substring($equalToIndex, $curlyLastIndex))
## #set ($tempCity = $city)
$util.qr($sanitisedListOfCities.add($tempCity))
#end
$util.toJson($sanitisedListOfCities)
The response that I am getting:
{
"data": {
"getCitiesForGroups": "[null, null, null, null, null, null]"
}
}
However, when I use the line #set ($tempCity = $city) and comment out the line above it, I get the following response:
{
"data": {
"getCitiesForGroups": "[{city=jaipur}, {city=mumbai}, {city=delhi}, {city=sheros}, {city=jalandhar}, {city=bengaluru}]"
}
}
Which means $city has the value {city=jaipur}, so which I want to sanitize and add it to ($sanitisedListOfCities) and return it as the response.
But I am getting null as the result substring method.
So how can I sanitize the response from DB and return it?
I had a similar issue and this is how I solved it.
First, your Graphql schema should look like,
type Query {
getCitiesForGroups: cityList
}
type cityList {
cities: [String]
}
Your request mapping template,
{
"version": "2018-05-29",
"statements": [
"select DISTINCT LOWER(city) as city from public.Groups"
]
}
And finally your response mapping template
#set($cityList = [])
#set($resMap = {})
#if($ctx.error)
$utils.error($ctx.error.message, $ctx.error.type)
#end
#foreach($item in $utils.rds.toJsonObject($ctx.result)[0])
$util.qr($cityList.add($item.city))
#end
$util.qr($resMap.put("cities", $cityList))
#return($resMap)
Expected response(complete)
{
"data": {
"getCitiesForGroups": {
"cities": [
"jaipur",
"mumbai",
"delhi",
"sheros",
"jalandhar",
"bengaluru"
]
}
}
}
I hope this helps you.
I'm trying to get a specific item from a table.
My DynamoDB table name is table and I have:
Name PK | Number<br/>
S: Juan | S: #####
When I try to run in Lambda I don't get any Item when it really exist one with that name... any idea why it's like that?
AWS = require("aws-sdk"),
DDB = new AWS.DynamoDB({
region: "REGION",
}),
lookup_name_str = name //From Intent variable,
params = {
TableName: "table",
KeyConditionExpression: "name = :v1",
ExpressionAttributeValues: {
":v1":{
"S": lookup_name_str
}
},
FilterExpression: 'contains(nomColaborador,:v1)',
ProjectionExpression: "Number"
};
console.log(params);
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.scan(params, function(err, data){
if(err){
throw err;
}
if(data.Items && data.Items[0] && data.Items[0].Number){
console.log("There is a Name with that number");
console.log(data.Items[0]);
my_response.statusCode = 200;
my_response.body = {
"sessionAttributes": {
"extension_str": data.Items[0].Number.S,
"nomColaborador": event.currentIntent.slots.Name
},
"dialogAction":{
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": data.Items[0].Number.S
}
}
};
The main problem here is that you are doing a scan. KeyConditionExpression isn't a parameter of a scan request. If you are requesting a single item by key you want to use getItem. If you need to query data by partition key and an optional sort key you should use query.
With that all said, when you do a scan, or put a filter on a query, you really need to be sure to page through the data. You will often find that you'll get a response with no data, but a paging key to make another call.
I have a column in DynamoDB table which will be of the following type:
{"History": {"L": [{"M": {"id": {"S": "id"}, "Flow": {"L":[{"S": "test2"}]},"UUID": {"S": "1234"}}}]}}
Column History is of type 'List' in which each list element is a map with 3 values - id (string), Flow (List), uuid (String)
My code would trigger update-item multiple times and all I want is, given the same id and uuid, new values are to be appended into the Flow list without disturbing anything else.
I have referred the documentation but unable to figure out how to write the UpdateExpression.
My existing code is as below:
response_update = client.update_item(
TableName = 'tableName',
Key = {
'k1': {
'S': 'v1'
},
'k2': {
'S': 'v2'
}
},
UpdateExpression="SET History=list_append(if_not_exists(History, :empty_list), :attrValue)",
ExpressionAttributeValues = {":attrValue" :{"L":[ { "M" : { "id" : { "S" : "123" }, "UUID" : { "S" : "uuid123" }, "Flow" : { "L" : [ { "S" : "now2" } ] } } } ]},":empty_list":{"L":[]}})
With this code, each time I trigger the update function, a new element in getting appended in History list. Instead, I need my desired string to be appended to the Flow list.
Please let me know how the expression should be.
I am working on an application that uses DynamoDB.
Is there a way where I can create a GSI with multiple attributes. My aim is to query the table with a query of following kind:
(attrA.val1 === someVal1 AND attrB.val2 === someVal2 AND attrC.val3 === someVal3)
OR (attrA.val4 === someVal4 AND attrB.val5 === someVal5 AND attrC.val6 === someVal6)
I am aware we can use Query when we have the Key Attribute and when Key Attribute is unknown we can use Scan operations. I am also aware of GSI if we need to query with non-key attributes. But I need some help in this scenario. Is there a way to model GSI to suit the above query.
I have the below item (i.e. data) on my Movies tables. The below query params works fine for me.
You can add the third attribute as present in the OP. It should work fine.
DynamoDB does support the complex condition on FilterExpression.
Query table based on some condition:-
var table = "Movies";
var year_val = 1991;
var title = "Movie with map attribute";
var params = {
TableName : table,
KeyConditionExpression : 'yearkey = :hkey and title = :rkey',
FilterExpression : '(records.K1 = :k1Val AND records.K2 = :k2Val) OR (records.K3 = :k3Val AND records.K4 = :k4Val)',
ExpressionAttributeValues : {
':hkey' : year_val,
':rkey' : title,
':k3Val' : 'V3',
':k4Val' : 'V4',
':k1Val' : 'V1',
':k2Val' : 'V2'
}
};
docClient.query(params, function(err, data) {
if (err) {
console.error("Unable to read item. Error JSON:", JSON.stringify(err,
null, 2));
} else {
console.log("GetItem succeeded:", JSON.stringify(data, null, 2));
}
});
My Data:-
Result:-
GetItem succeeded: {
"Items": [
{
"title": "Movie with map attribute",
"yearkey": 1991,
"records": {
"K3": "V3",
"K4": "V4",
"K1": "V1",
"K2": "V2"
}
}
],
"Count": 1,
"ScannedCount": 1
}