I'm starting out using some of the managed services in AWS. One thing that seems like it should be easy, is to use the API gateway to secure and expose calls to DynamoDB.
I've got this working. However, it seems a little clunky. DynamoDB returns something like this:
{
"id":{"N":"3"}
// Lots of other fields
}
When really I (and most other consumers out there) would like something like this:
{
"id":"3"
// Lots of other fields
}
The way I see it, I've got two options.
1) Add a response mapping field by field in the AWS API UI. This seems laborious and error prone:
#set($inputRoot = $input.path('$'))
{
"Id": "$elem.Id.N"
// Lots of other fields
}
2) Write a specific lambda between API Gateway and Dynamo that does this mapping. Like https://stackoverflow.com/a/42231827/2012130 This adds another thing in the mix to maintain.
Is there a better way? Am I missing something? Seems to be so close to awesome.
const AWS = require('aws-sdk');
var db = new AWS.DynamoDB.DocumentClient({
region: 'us-east-1',
apiVersion: '2012-08-10'
});
You can use ODM like dynogels,
https://github.com/clarkie/dynogels
We use that heavily without dealing with dynamodb syntaxes.
This brings lambda and language in the mix, but it is much easier to handle when an object grows larger to perform the mapping.
Hope this helps.
There's another option added today. It'll still involve a lambda step, but..
"The Amazon DynamoDB DataMapper for JavaScript is a high-level client for writing and reading structured data to and from DynamoDB, built on top of the AWS SDK for JavaScript."
https://aws.amazon.com/blogs/developer/introducing-the-amazon-dynamodb-datamapper-for-javascript-developer-preview/
Related
https://boto3.amazonaws.com/v1/documentation/api/1.9.42/reference/services/dynamodb.html#service-resource
For Dynamodb webservice we have
dynamodb = boto3.resource('dynamodb') and
client = boto3.client('dynamodb')
Both have Query and Scan methods. Are there any pros and cons using Query on a client object vs Resource object?
You can actually use both to interact with AWS API. But there are some differences.
Client is a low level service access. Resource is higher level object oriented API.
Most of the times, even on aws docs you'll see client is being used. Unless my requirements need something else, I stick to the official docs.
To know more in depth you can see this and this.
Client is a low level interface where you must work with DynamoDB items using DynamoDB JSON:
{"id":{"S":"some-id"}}
Having to work with the lower level client is a little more difficult to construct ConditionExpressions etc...
Resource is a high level interface where it abstracts the DynamoDB JSON and allows you to use native JSON:
{"id":"some-id"}
This simplifies how your construct your conditions but also allows parsing the result set easier, without having to call utility functions such as unmarshall.
Performance wise there is no difference. Personally I like using the Resource level.
Is there a simple way to retrieve all items from a DynamoDB table using a mapping template in an API Gateway endpoint? I usually use a lambda to process the data before returning it but this is such a simple task that a Lambda seems like an overkill.
I have a table that contains data with the following format:
roleAttributeName roleHierarchyLevel roleIsActive roleName
"admin" 99 true "Admin"
"director" 90 true "Director"
"areaManager" 80 false "Area Manager"
I'm happy with getting the data, doesn't matter the representation as I can later transform it further down in my code.
I've been looking around but all tutorials explain how to get specific bits of data through queries and params like roles/{roleAttributeName} but I just want to hit roles/ and get all items.
All you need to do is
create a resource (without curly braces since we dont need a particular item)
create a get method
use Scan instead of Query in Action while configuring the integration request.
Configurations as follows :
enter image description here
now try test...you should get the response.
to try it out on postman deploy the api first and then use the provided link into postman followed by your resource name.
API Gateway allows you to Proxy DynamoDB as a service. Here you have an interesting tutorial on how to do it (you can ignore the part related to index to make it work).
To retrieve all the items from a table, you can use Scan as the action in API Gateway. Keep in mind that DynamoDB limits the query sizes to 1MB either for Scan and Query actions.
You can also limit your own query before it is automatically done by using the Limit parameter.
AWS DynamoDB Scan Reference
I am trying to create an app that uses AWS Services, I already use Cognito plugin for flutter but can't get it to work with DynamoDB, should I use a lambda function and point to it or is it possible to get data form a table directly from flutter, if that's the case which URL should I use?
I am new in AWS Services don’t know if is it possible to access a dynamo table with a URL or I should just use a lambda function
Since this is kind of an open-ended question and you mentioned Lambdas, I would suggest checking out the Serverless framework. They have a couple of template applications in various languages/frameworks. Serverless makes it really easy to spin up Lambdas configured to an API Gateway, and you can start with the default proxy+ resource. You can also define DynamoDB tables to be auto-created/destroyed when you deploy/destroy your serverless application. When you successfully deploy using the command 'serverless deploy' it will output the URL to access your API Gateway which will trigger your Lambda seamlessly.
Then once you have a basic "hello-word" type API hosted on AWS, you can just follow the docs along for how to set up the DynamoDB library/sdk for your given framework/language.
Let me know if you have any questions!
-PS: I would also, later on, recommend using the API Gateway Authorizer against your Cognito User Pool, since you already have auth on the Flutter app, then all you have to do is pass through the token. The Authorizer can also be easily set up via the Serverless Framework! Then your API will be authenticated at the Gateway level, leaving AWS to do all the hard work :)
If you want to read directly from Dynamo It is actually pretty easy.
First add this package to your project.
Then create your models you want to read and write. Along with conversion methods.
class Parent {
String name;
late List<Child> children;
factory Parrent.fromDBValue(Map<String, AttributeValue> dbValue) {
name = dbValue["name"]!.s!;
children = dbValue["children"]!.l!.map((e) =>Child.fromDB(e)).toList();
}
Map<String, AttributeValue> toDBValue() {
Map<String, AttributeValue> dbMap = Map();
dbMap["name"] = AttributeValue(s: name);
dbMap["children"] = AttributeValue(
l: children.map((e) => AttributeValue(m: e.toDBValue())).toList());
return dbMap;
}
}
(AttributeValue comes from the package)
Then you can consume dynamo db api as per normal.
Create Dynamo service
class DynamoService {
final service = DynamoDB(
region: 'af-south-1',
credentials: AwsClientCredentials(
accessKey: "someAccessKey",
secretKey: "somesecretkey"));
Future<List<Map<String, AttributeValue>>?> getAll(
{required String tableName}) async {
var reslut = await service.scan(tableName: tableName);
return reslut.items;
}
Future insertNewItem(Map<String, AttributeValue> dbData, String tableName) async {
service.putItem(item: dbData, tableName: tableName);
}
}
Then you can convert when getting all data from dynamo.
List<Parent> getAllParents() {
List<Map<String, AttributeValue>>? parents =
await dynamoService.getAll(tableName: "parents");
return parents!.map((e) =>Parent.fromDbValue(e)).toList()
}
You can check all Dynamo operations from here
Simple question, but I suspect it doesn't have a simple or easy answer. Still, worth asking.
We're creating an implementation for push notifications using AWS with our Web Server running on EC2, sending messages to a queue on SQS, which is dealt with using Lambda, which is sent finally to SNS to be delivered to the iOS/Android apps.
The question I have is this: is there a way to query SNS endpoints based on the custom user data that you can provide on creation? The only way I see to do this so far is to list all the endpoints in a given platform application, and then search through that list for the user data I'm looking for... however, a more direct approach would be far better.
Why I want to do this is simple: if I could attach a User Identifier to these Device Endpoints, and query based on that, I could avoid completely having to save the ARN to our DynamoDB database. It would save a lot of implementation time and complexity.
Let me know what you guys think, even if what you think is that this idea is impractical and stupid, or if searching through all of them is the best way to go about this!
Cheers!
There isn't the ability to have a "where" clause in ListTopics. I see two possibilities:
Create a new SNS topic per user that has some identifiable id in it. So, for example, the ARN would be something like "arn:aws:sns:us-east-1:123456789:know-prefix-user-id". The obvious downside is that you have the potential for a boat load of SNS topics.
Use a service designed for this type of usage like PubNub. Disclaimer - I don't work for PubNub or own stock but have successfully used it in multiple projects. You'll be able to target one or many users this way.
According the the [AWS documentation][1] if you try and create a new Platform Endpoint with the same User Data you should get a response with an exception including the ARN associated with the existing PlatformEndpoint.
It's definitely not ideal, but it would be a round about way of querying the User Data Endpoint attributes via exception.
//Query CustomUserData by exception
CreatePlatformEndpointRequest cpeReq = new CreatePlatformEndpointRequest().withPlatformApplicationArn(applicationArn).withToken("dummyToken").withCustomUserData("username");
CreatePlatformEndpointResult cpeRes = client.createPlatformEndpoint(cpeReq);
You should get an exception with the ARN if an endpoint with the same withCustomUserData exists.
Then you just use that ARN and away you go.
I have been trying to setup a resource using AWS API Gateway but I can't seem to find a way to either set or access matrix parameters.
I want to be able to set up a resource similar to the following -
GET /image;height=750;width=1000;format=png
Is it possible ?
You need to configure the setup to use Query Parameters.
You do this in the Method Request area of a method configuration from within the console:
https://console.aws.amazon.com/apigateway/home?region=<region-id>#/restapis/<api-id>/resources/<resource-id>/methods/<method-type>
You can also do this use the AWS API Gateway HTTP API putMethod endpoint, or the AWS#APIGateway#putMethod call in any of their SDKs.
API Gateway currently does not support matrix parameters. As a workaround, you could use query parameters as already mentioned and parse them in your backend.
Best,
Jurgen
I realize that this is a very old question. Leaving my response in case someone has a similar challenge
There are multiple ways to set this up. It all comes down to the context in which the API is intended to be used.
While query parameters will solve the problem, they are not the best suited for representing a resource. They fit well with scenarios that involve filtering. If this API is intended to be used as a source for <img /> tags on the UI, this pattern GET .../images/{widthxheight}/{imageName}.{extension} can be used.
Ex: GET .../images/200x400/sponge-bob.png
However, if the intent is for this API to be used for the purpose of looking up. the below definition can be used -
POST .../image-results
Content-Type: application/json
{
"name":"sponge bob",
"height": 400,
"width": 200,
"format": "png"
}