I have two seprate apps each has seprate AWS cognito userpool appsync api but has shared dynamodb. I want to create subscription for chat feature where app 1 (client app) and app 2 (admin app) will communicate. Is it possible please advise.
I have followed this article from aws:
enter link description here
need advise how would that work in my case with two different apps.
I think you first need to take a step back and separate the problem from the technology. You want to create a chat app, how would you build this? Do you need a data store or a queue? First think in abstractions then look what technology fits those abstractions best. If you start with the technology first you will most likely end up with a sub-par solution.
If you really want to go for the technology first approach you could think of storing chats in DynamoDB(DDB) and using DDB streams to update the subscription. It can work but it will most likely be expensive.
Related
Lets assume I have a client who has plethora of data related to railways(Signals, tracks, train timings, hazard. offers etc). There are various internal department in railways which wants that data. Just like various weather websites get data from weather department and show that data on their website. Similar is my requirement that I want to share the data securely with other department and services. I want to look at best method to share the data to other services as quickly as possible when the data is available.
Possible Solution
API based: Create API for each department and share them data via API. This has its own pros and cons. This is something which came to my mind but we would have to create lot go API's I was looking if Azure and AWS has any other service which can do the same.
Azure based solution: I am looking for help in this if Azure and services it provides can help. Service bus, Event Grid, Event Hub etc can these be of any use?.
AWS based solution: Is there any service in AWS which can help here?. I dont have much exposure to AWS.
Any other solution ?
I have a fair idea that this could be built using API's but I am looking if I can get this done using cloud platforms like Azure and AWS> This will help in better integration of the product and can scale.
i just arrived on this architecture, am doing a lot of research and i understood how it work in general but it's all theorical.
I decided to separate each step for the development of this architecture to start implementing so i can understand better these steps.
The first that i wanted to learn was the tenant provisioning, i wanted to apply it on AWS to mirror a production software example.
So, starting on that the common AWS service that i see most people using is AWS Cognito, but it's not clear in my mind the steps of the implementation, like how should i get the tenant data to onboard him in my app? Assuming it's tier based.
Should i have one database to store all tenants data separate from the application database?
I want to use microservices on this one because i think is better to onboard the tenant with different tiers and much more benefits.
Which AWS services should i use to make this process work? I'm not really asking about the implementation itself but a path to understand which services to use and how it connects with each other.
I hope i was clear about my doubts, english is not my mother tongue, sorry about that!
You are thinking in the right direction. However, there are decisions you need to make before diving into any saas service stack. I would start with
Planning my infrastructure - how many tenants/group.
the kind of tenant onboarding system you want
How will tenants onboard their users and manage authorization/authentication
Multitenant architecture, which needs to account for several things at the least like - DB model, shared vs isolated, data privacy, design keeping in mind industry data security standards
what will be your tenant deployment model. Remember one of the disadvantages of multitenancy is also slow time to market.
Your API stack needs to account for which apis needs to be multitenant and which are generic product offerings.
operational tool to monitor app health, client analytics.
how will you meter and bill the client and other non-functional decisions.
AWS offers good documentation to get started here : https://aws.amazon.com/blogs/apn/building-a-multi-tenant-saas-solution-using-aws-serverless-services/
I just want to avoid use of custom/manual resolvers in appsync completely. So I'm using Amplify to setup GraphQL appsync API in my app. I'm doing all the stuffs by changing schema.graphql and amplify push.
I have 2 questions :
1. What are the limitations and what problems I'm going to face in future?
2. Can graphql subscriptions get update when app is not running(like user should be notified)?
tons of business logic will be exposed on the client side code.
I think for push notifications you would still have to go via external integrations like FCM/APNS. Multiple integration options are available in SNS
Just to preamble these answers, the fact that you use an amplify generated graphQL and resolvers doesn't stop you from later including custom resolvers and pipeline functions - it's just that you need to learn quite a bit about where to include them in the backend file structure of amplify.
1. What are the limitations and what problems I'm going to face in future?
This depends on how well your applications use-case matches the graphQL schema design and if your application is relatively self-contained. Amplify becomes more complex when your application needs to talk to other back-end systems, you'll need to start using DynamoDB triggers to notify other state machines/event bridge/SNS or similar services.
As mentioned none of these problems are crippling, you can deal with them later but it will be a step up in the AWS knowledge required to implement them.
For small high-volume/availability apps Amplify and DynamoDB as-it-comes is great. If your application matures into many micro-services and sites then you'll need to learn quite a bit more AWS to make them play together well. Amplify does determine your DynamoDB on a table per object basis and you'll probably be stuck with (paying for) that. Think hard about if you ever might want to go to a different optimised data source (RDS or single dynamo table) to reduce the number of queries required to fulfil your graphQL requests.
2. Can graphql subscriptions get update when app is not running(like user should be notified)?
No. Anurag mentions SNS which would be a good option to out-app notify users, best to blend subscriptions and another service.
I am trying to write a Kafka connector to fetch data from the facebook. The problems are,
How to fetch data from facebook through their API without exceeding the limit of API hit provided by facebook? The connector should call facebook API for data after a specific time interval so that the number of hits won't exceed.
Each user can hit the facebook API with their Access Token so users can't share the same topic partition. So how to handle this scenario. Do we have to create one partition for each user?
I read a few guides and blogs to understand Kafka connect and write a connector.
Confluent- https://docs.confluent.io/current/connect/index.html
Kafka Documentation- https://kafka.apache.org/documentation/#connect
Conceptually It gave me an idea about what is Kafka connect, how it works and what are the important classes to write a Kafka connector. But still, I am confused that practically how to write and run a connector. I tried to find step by step development guide but didn't get.
Any tutorial or pdf If you could suggest which have detailed step by step development guide to write and run Kafka connector.
The only "official guide" is in those links you have
https://docs.confluent.io/current/connect/devguide.html#developing-a-simple-connector
I personally have no experience with the Facebook API, but I assume it uses REST, so you could make start by forking the kafka-connect-rest project, but the simplest answer to not exceed the limit would be to not send more requests than you are allowed within a given time period (add a timer to the code that waits between requests)
Also, one connector would only have one set of access keys. How you create the ConnectRecord objects to ultimately partition the records is up to you, but I don't think having an access key per user will scale very well. It might make more sense to have one key tied to one application, then each user will accept that that application has access to read certain details from their account.
I'm new to AWS, so apologies in advance if this question is missing some important considerations, or has incorrect assumptions.
But basically I want to implement a service on AWS to store and retrieve data from multiple clients, which may be Android apps, Windows applications, websites etc. The way I've considered doing this is via a RESTful service using API Gateway front end, with a Lambda back end and maybe an S3 bucket to hold the data.
The basic requirements are:
(1) Clients can publish data to the server, where it is stored, perhaps with some kind of key/value structure.
(2) Clients can retrieve said data by key.
(3) If it is possible, clients to be able to subscribe to events from the service, so that they are notified if the value of a piece of data changes. This would avoid the need to poll the service, which would presumably start racking up unnecessary charges if the data doesn't change often.
Any pointers on how to get started with this welcome!
Creating a RESTful API on top of Lambda and API Gateway is one of the main use cases for this architecture. You can think of Lambda functions as controllers with methods and API Gateway as a router that forwards requests to functions based on the URL pattern. There are many frameworks and approaches that can help out here if you don't want to write from scratch:
Lambdasync
https://medium.com/#fredrikanderzon/create-a-rest-api-on-aws-lambda-using-lambdasync-e46c68f8043f
Serverless
https://serverless.com/framework/docs/providers/aws/events/apigateway/
Swagger
https://cloudonaut.io/create-a-serverless-restful-api-with-api-gateway-swagger-lambda-and-dynamodb/
As far as event subscriptions go (requirement #3) you can model this in many datastores, certainly in a relational/SQL database, with a table like this:
Subscription (key_of_interest, user_id, events_of_interest)
I'm leaving out data types for you to figure out, but you get the idea hopefully. After each data modification on a particular key, see if that key is of interest in the subscription table, then wire up a response to the user's who indicated interest. The details of this of course depend on your particular requirements. A caution though: this approach will increase the cost of data modifications because of the additional overhead needed to process subscriptions.
EDIT: One other thing I forgot. S3 is better suited for non-structured data (think 'files'). For relational databases, checkout RDS. For a simple NoSQL database you might use DynamoDB, or host your own NoSQL database of choice on an EC2 instance.