Using:
AppSync, DynamoDB, and Lambda
So I am a bit stuck on how to integrate AppSync within Amplify in React Native. I have an existing API within AWS AppSync that I created on the console. This API has several different models like: Users, Videos, VideoComments, etc. Some of the objects within these models have custom mapping templates and resolvers that are really important for the application.
For Example, this is a quick look at what we have schema wise.
type User {
userId: Id!
name: String
uploadedVideos(limit: Int, nextToken: String): VideoConnection
etc etc
}
type Video {
videoID: Id!
object: S3Object
userId: Id!
uploadedBy: User
}
We have a resolver that runs with a simple getVideo query that will retrieve the uploadedBy attribute by using the userId, and retrieves all the necessary information of that user.
Additionally, the data sources (dynamoDb tables) we created for the models have primary keys, and some have sort keys. Like a VideoLikes table keeps track of who liked a video and to avoid duplicates the primary key is the VideoID and the sort key is UserID. This is just a minor example, we have other places where we do this to also have access to LSIs.
When I started using Amplify, I tried recreating the AppSync API because I liked how powerful the CloudFormation capabilities were with the different staging environments. However, I noticed the DynamoDB for models were automatically defined and were automatically set to an id as a primary key. We use the LSIs to help sort by certain values, like for a video if we wanted to sort by the number of likes, or comments, so unfortunately this would not work for us. So when I noticed this, I used the "Codegen" command from my original AppSync API and ran into the issue where my resolvers and mapping templates were not copied down with the schema, queries, mutations, and subscriptions making most of the queries fail because those data sources were missing.
So my question is:
Is there a way to integrate/use EVERYTHING from my exiting AppSync API within my React Native application? This includes the custom resolvers and mapping templates.
IF NOT
Is there a way to set a primary and sort key for the DynamoDB of the Models when creating an API directly within the Amplify CLI?
IF NOT
Is there another way to have data sorted efficiently within DynamoDB without using LSIs and GSIs? If the models automatically generate tables with GSIs, this can be problematic because I know GSIs are a bit more expensive so I would like to avoid those as much as possible. Is there another service that will sort the data that can be used within AppSync from DynamoDB?
Any help will be appreciated, thank you.
Is there a way to integrate/use EVERYTHING from my exiting AppSync API within my React Native application?
Yes, you can deploy your API however you want and then use it using amplify client tools. You can always use the Ampify CLI's codegen features without deploying the API via the amplify api category. You can use custom stacks to define custom resolvers that are not generated by #model but this will deploy a new API with the same structure as your existing API.
Is there a way to set a primary and sort key for the DynamoDB of the Models when creating an API directly within the Amplify CLI?
Soon. There is an RFC here https://github.com/aws-amplify/amplify-cli/issues/1062 and implementation is in PR here https://github.com/aws-amplify/amplify-cli/pull/1463.
Is there another way to have data sorted efficiently within DynamoDB without using LSIs and GSIs?
No but you can use index overloading to make 1 index store and sort multiple different conceptual object types. TBH this is a complex subject but here is a good place to start https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-gsi-overloading.html.
Related
i have the following question.
I have in my app an Table called Networkoperator
This table should be filled on creation. I have an JSON file with the data in it that i need to insert.
How is this achievable?
I was thinking of using an CDK that triggers an lambda only on creation
The Amplify documentation has a plugins page that lists a bunch of 3rd party plugins. There are two plugins that help seed DB data, but they both appear a bit out of date.
I'd be inclined to make a custom category (amplify add custom) and make it a dependency of the database table (ideally created and managed by Amplify). Then you can do anything you like in the 'build' step of the custom category's package.json. The aws-cdk-dynamodb-seeder package looks like it would do the trick (but was also published 2 years ago). Within the custom category, you can pull in the DB table name, etc, to avoid hardcoding.
By default AWS Amplify transformers creating tables per each graphql type.
But according DynamoDB documentation it's best practice to
Keep tables few as possible
Keep often queried together entries within a same table
I have an impression Amplify way of doing things stays in contradiction with the statement above.
I am new to both NoSQL and Amplify
Can someone suggest ways to address those issues?
I think we're in a bit of a transition or gray area here. I'm very new to Amplify and have been investigating moving to a single-table design as there are sources (below) that indicate that it's always be there but you'd have to write everything in VTL templates. But in 2020 they released direct lambda resolver support: https://youtu.be/EOQqi6Yun7g?t=960 (clip)
However, it seems like you lose access to the #auth directive (and probably others because you're no longer going to use #model) along with a lot of the nice out-of-the-box functionality that's available with Amplify's multi-table approach.
At this point, being that I'm developing a new app, I'm going to stick with the default multi-table design to hasten the process of getting the app functional.
Trying to implement the single-table design seems to go against what the Amplify team recommends and requires more manual work. You'd have to manually create custom lambda functions (AppSync) and code queries to DynamoDb for each data access element and manage authorization through some other means which I'm not aware of at this time. Maybe someone can chime in here...
Single table vs multi table info
Using Amplify with single table:
https://youtu.be/EOQqi6Yun7g
Single vs Multi Clip:
https://youtu.be/1WF_wped808?t=1251 (clip)
https://www.alexdebrie.com/posts/dynamodb-single-table/ (towards bottom)
https://youtu.be/EOQqi6Yun7g?t=1288 (clip)
Example single table design by Alex Debrie:
https://gist.github.com/dabit3/96dc51e688b18a7d40fc534331758c56
More Discussion:
https://stackoverflow.com/a/56438716/1956540
Basic Setup steps
I setup a single table by following the below instructions. Again, you don't use #models for this. Also, I think you have to include a type query {} in your schema for it to compile, but I could be wrong here.
So the basic steps are:
Create a single table (amplify add storage)
amplify push
Create your schema in the schema.graphql file.
Create supporting lambda function (amplify add function)
Note: if you look at the example here, I believe you can create an entry point to routes to all other methods: https://gist.github.com/dabit3/96dc51e688b18a7d40fc534331758c56#lambda
Add the DynamoDb query code in the function.
amplify push
Complete steps for Setting up a single Table:
https://catalog.us-east-1.prod.workshops.aws/workshops/53b10bf8-2271-4ab4-bfd2-39e878a90dc8/en-US/lab2/1-vtl (both "Connecting to an existing DynamoDB table" and "Direct Lambda Resolver" steps)
Not trying to be negative about Amplify, it is awesome, I love what they are doing with this product. I just think it's very new to everyone and I'm hoping this post is no longer valid next year and we continue to see great progress from the team.
I am trying to do something that would be relatively simple for a relational database but I don't know how to do it for a nonrelational one.
I am trying to make a simple task web app on AWS where people can post their tasks.
I have a table called tasks which uses the userid from the auth token provisioned by AWS Cognito. I am wondering how I can return the user information. I do not want to rely on Cognito by simply calling it every time a user sends a request. So, my thought would be to create another table to store all of the user information. That, however, is not a very nonrelational way of doing things since JOINS are so bad.
So, I was wondering if I should do any of the following
a) Using RDS instead
b) Not use Cognito and set up my own Auth system
c) Just doing the JOIN with a table containing all of the user info
d) Doing the request to Cognito each time
Although I personally like the idea of cognito, at this time it has some major drawbacks...
You can not backup / restore a user pool without loosing their password, also you have to implement your own backup/restore.
A way around is to save the user password in a cognito custom attribute.
I expected by using api gateway/lambda authorizer to have all the user data in the lambda context but its not there. Or am indoing something wrong with api gateway template mapping 😬
Good thing api gateway/lambda authorizer, can be cached by up to an hour, wont call the authorizer function again which seems like a top feature.
Does not work well with cloudformation, with every attribute update it recreates the user pool without restoring the users, thus loosing the users.
I used it only in one implementation and ended up duplicating the users in DynamoDB as well.
I'm avoiding it ever since. I wish they solve these issues as it looks like a service to be included with every project saving lot of time.
Reading your post I asked myself the same questions and not sure the answer either 😄
Pricing seems fair.
The default 5 requests/second to get user info seems strange as it woukd be consumed by one page load doing multiple ajax api requests .
For this in DynamoDB, there is no need for another table. If the access patterns dictate you store the information in another object, then so be it, but more than likely it should be in the same table. Sounds like you need two different item types in the same table.
For the task PK of userid and SK of task::your-task-id. This would allow you to get all of a user's tasks easily or even a specific task very easily if you knew the task ID. You might even have an attribute that is a timestamp and then have a GSI that is the userID as the PK and the timestamp as the SK. then you could use the begins_with operator on the SK and "paginate through all of the user's tasks that are in the month of 2019-04".
For the user information, have the userID be the PK and the SK be user_info and attributes be the user's information.
The one challenge for this is if you were to go to extremes and one single user is doing thousands of ops per second. e.g. "All tweets by very popular celebrity". If you have such a use case there are ways around that as well, e.g. write sharding. These are just examples for you to play with. Without knowing all your access patterns, I cannot model everything you might want to do. I highly recommend you go watch this presentation from reInvent 2018.
According to the AWS Amplify documentation:
"objects annotated with #model are stored in Amazon DynamoDB";
"a single #model directive configures ... an Amazon DynamoDB table"; and
one can "push updated changes with amplify push".
It seems clear that amplify push creates a DynamoDB table for each #model.
My questions relate to schema updates:
I imagine that adding/removing a model or adding/removing a field in a model works by updating the schema document and then running amplify push. Is that right?
How does one rename a model or a field? How would amplify push know to rename vs. drop the old and add the new?
How does one implement a migration that requires some business logic, e.g., to update the contents of existing rows? Doing this without Amplify has already been addressed but it is unclear whether that would conflict with something that amplify push might try to do.
DynamoDB is schema-less, and doesn't care about your application schema as long as you don't try to change its hash key or range key
Therefore, nothing really happens on the datastore side. If you drop a key and add a new one in your schema, then your application will start to search and write data to the new key. Old key will simply be ignored from now on, but existing data will be kept in the datastore.
If you want to rename a key, then you would have to migrate the data by yourself through mass update on the table. There are many ways to do it, the simpliest one being scaning the table and performing updates on found items.
Have you tried compiling the schema with this:
amplify api gql-compile
Try running
amplify codegen models
before doing the
amplify push
I am very new to AWS. As the first step I am creating an eCommerce application on my personal interest to give the demo of this application to my colleagues.
I am implementing 'Order' part. For this, I am thinking of moving the data from one table to other. I.e Once the user add the product to cart , it will saved in Cart table in dynamo-db and in cart screen when the user clicks on 'Order'button/Link, the same data as it is in cart table should be moved to Order table and the cart should be empty So, the order can be confirmed.
How could I implement it? Not sure the method I am thinking is right if there any other method to accomplish Order functionality.
The answer to this is really going to depend on your architecture and stack - and even within that you have lots of options.
In a serverless way, i.e. from a static html page with no server-side backend, you could create a lambda function in the supported language of your choice and with the proper IAM role, to move the data from one table to the other - your html page could call it via an API call, and I would suggest you use AWS API Gateway to expose an api endpoint that then calls the lambda function.
If on (one of the other many) other-hands, say you were using ASP.net or PHP on the server side, you could use the AWSSDK to talk to the dynamodb directly and accomplish the same thing.
Besides these two options there are many, many alternatives and variations - and with all of the options you are also going to need to deal with authentication/security to make sure no one can make calls to your database/service that they aren't permitted to - perhaps not important for your demo application, but will definitely be an issue if/when you go live.