REST API with complex queries - django

I don't fully understand how to make complex queries with REST API on the backend, and a backbone web app on the frontend. Say I have a user table that has a relation with a user_group table:
user.group_ref => user_group.id
If I do a GET on /api/v1/user/1/?format=json it will do something like SELECT * FROM users WHERE id = 1. Right, ok, now... how about if I wanted to JOIN with my user_group where user.group_ref = user_group.id to instantly have access to the data on user_group. I don't want to do an extra query to go and fetch that data.
Maybe I got the whole idea wrong ... Is there a simpler way?

REST is a set of conventions. It doesn't provide for automatic query mapping, so you need to define a service endpoint, and then implement it to return whatever you want it to.
In your case a typical way of composing the URL would be something like:
GET /groups/(groupid)/users
Which is to say "give me all users belonging to this group". Alternatively:
GET /users?group=(groupid)
Which in style is less "RESTful", but doesn't unnecessarily promote group as a top-level resource.
Either way, Backbone doesn't provide an OOTB way for populating collections from more complex resources. For anything beyond simple CRUD you'll have to implement the service call yourself, or create a separate read-only collection with url that maps to your service. Something like:
var UserGroupCollection = Backbone.Collection.extend({
url: function() { return "groups/" + this.options.groupId + "/users"; }
});
var group = new UserGroupCollection({groupId:1});
group.fetch();

The REST API can be implemented separately, without any relationship to the front end, it can be designed as you need.
In your case, if you are always going to get the user_group.id with the query for a user, then you should change your SQL to JOIN statements permanently: SELECT * FROM users WHERE id = 1 JOIN user_group WHERE user.group_ref=user_group.id.
If you need both queries with and without user_group info. You can design two REST methods, such as (GET /api/v1/user/1?format=json) for without group info, and (GET /api/v1/userwithgroup/1?format=json) for ones with group info.
And on the Backbone, you could have two different Models representing the two.
If you don't have millions of users in the table, the SQL with group info should be quite fast and it would be easier to just always have group info with it.

There is a python library called slubmber (http://slumber.in/) which is built on top of requests for the explicit purpose of REST APIs.

Related

How to structure AWS DynamoDB Table with Cognito

I am trying to do something that would be relatively simple for a relational database but I don't know how to do it for a nonrelational one.
I am trying to make a simple task web app on AWS where people can post their tasks.
I have a table called tasks which uses the userid from the auth token provisioned by AWS Cognito. I am wondering how I can return the user information. I do not want to rely on Cognito by simply calling it every time a user sends a request. So, my thought would be to create another table to store all of the user information. That, however, is not a very nonrelational way of doing things since JOINS are so bad.
So, I was wondering if I should do any of the following
a) Using RDS instead
b) Not use Cognito and set up my own Auth system
c) Just doing the JOIN with a table containing all of the user info
d) Doing the request to Cognito each time
Although I personally like the idea of cognito, at this time it has some major drawbacks...
You can not backup / restore a user pool without loosing their password, also you have to implement your own backup/restore.
A way around is to save the user password in a cognito custom attribute.
I expected by using api gateway/lambda authorizer to have all the user data in the lambda context but its not there. Or am indoing something wrong with api gateway template mapping 😬
Good thing api gateway/lambda authorizer, can be cached by up to an hour, wont call the authorizer function again which seems like a top feature.
Does not work well with cloudformation, with every attribute update it recreates the user pool without restoring the users, thus loosing the users.
I used it only in one implementation and ended up duplicating the users in DynamoDB as well.
I'm avoiding it ever since. I wish they solve these issues as it looks like a service to be included with every project saving lot of time.
Reading your post I asked myself the same questions and not sure the answer either 😄
Pricing seems fair.
The default 5 requests/second to get user info seems strange as it woukd be consumed by one page load doing multiple ajax api requests .
For this in DynamoDB, there is no need for another table. If the access patterns dictate you store the information in another object, then so be it, but more than likely it should be in the same table. Sounds like you need two different item types in the same table.
For the task PK of userid and SK of task::your-task-id. This would allow you to get all of a user's tasks easily or even a specific task very easily if you knew the task ID. You might even have an attribute that is a timestamp and then have a GSI that is the userID as the PK and the timestamp as the SK. then you could use the begins_with operator on the SK and "paginate through all of the user's tasks that are in the month of 2019-04".
For the user information, have the userID be the PK and the SK be user_info and attributes be the user's information.
The one challenge for this is if you were to go to extremes and one single user is doing thousands of ops per second. e.g. "All tweets by very popular celebrity". If you have such a use case there are ways around that as well, e.g. write sharding. These are just examples for you to play with. Without knowing all your access patterns, I cannot model everything you might want to do. I highly recommend you go watch this presentation from reInvent 2018.

How to move the data from one DynamoDB table to other DynamoDB table in the same region on click of HTML button

I am very new to AWS. As the first step I am creating an eCommerce application on my personal interest to give the demo of this application to my colleagues.
I am implementing 'Order' part. For this, I am thinking of moving the data from one table to other. I.e Once the user add the product to cart , it will saved in Cart table in dynamo-db and in cart screen when the user clicks on 'Order'button/Link, the same data as it is in cart table should be moved to Order table and the cart should be empty So, the order can be confirmed.
How could I implement it? Not sure the method I am thinking is right if there any other method to accomplish Order functionality.
The answer to this is really going to depend on your architecture and stack - and even within that you have lots of options.
In a serverless way, i.e. from a static html page with no server-side backend, you could create a lambda function in the supported language of your choice and with the proper IAM role, to move the data from one table to the other - your html page could call it via an API call, and I would suggest you use AWS API Gateway to expose an api endpoint that then calls the lambda function.
If on (one of the other many) other-hands, say you were using ASP.net or PHP on the server side, you could use the AWSSDK to talk to the dynamodb directly and accomplish the same thing.
Besides these two options there are many, many alternatives and variations - and with all of the options you are also going to need to deal with authentication/security to make sure no one can make calls to your database/service that they aren't permitted to - perhaps not important for your demo application, but will definitely be an issue if/when you go live.

Accessing a distant relation using $owner in loopback

I just started using loopback, and I am stuck with ACL. My database has a relation like so:
User has many tests and tests have many users ( many to many, in loopback I am using hasmanythrough).
Each test has several sections (one to many)
Each section has several question(one to many)
Now, I want to get all sections that a user has , or all questions that a user has. I know that using $owner needs a belongsTo in the respective model, but in my case that is not possible.
Is there any way to achieve this without having to completely write my own queries ?
Unfortunately the $owner role doesn't work as a filter, but as security access to end-points when an instance ID is specified; basically it only works when you perform a findById, but not when you perform a find.
Example:
GET /api/tests/ does nothing. The current user sees ALL The tests. No filtering is performed
GET /api/tests/{id} checks that the currently logged in userId corresponds to the userId in the test you are trying to retrieve. If the userIds match, then the user can view this particular test. if they do not match then you get an AUTHORIZATION_REQUIRED or ACCESS_DENIED error (I can't remember which).
as I just wrote in this question, you might want to look at creating a Mixin.

How to design Restful URIs with mapping based resources

I am using Django rest framework for writing some rest API in which I have two primary resources as Store and Product.
Now the URIs for these resources are like
List all the stores GET /stores/
List store with ID GET /stores/:id/
Add a new store POST /stores/
Update some attributes in a store PUT /stores/:id/
Same is applicable to products
List all the products GET /products/
List product with ID GET /products/:id/
Add a new product POST /products/
Update some attributes in a product PUT /products/:id/
Now i want to map some products to some stores like if store A is selling products with ID 1 to 100 and store B is selling products with ID 70 to 200.
With this i have got another resource as store_product_mappings
Now i can treat it as above and make URIs for this resource as
List all the store_product_mappings GET /store_product_mappings/
List store_product_mapping with ID GET /store_product_mappings/:id/
Add a new store_product_mapping POST /store_product_mappings/
Update some attributes in a store_product_mapping PUT /store_product_mappings/:id/
But now this mapping id isn't exposed to the consumers of this API. So in order to avoid this complexity i want to make a easilty understandable URI like
To list all the products inside a store GET /stores/:id/products/
To list a product inside a store GET /stores/:id/products/:product_id
To update a product inside a store PUT /stores/:id/products/:product_id
But according to this approach if i do post on this URI
POST /stores/:id/products/
It should create a new mapping resource and return the ID for that mapping resource, in the similar way it will expect the mapping id when i want to retrieve that resource back like
GET /stores/:id/products/:id
But i don't want to expose mapping ids as it's part of my internal system and consumers of API shouldn't need to worry about it. I am still looking for solutions for this kind of use case and resources in Django rest framework.
Let me know if someone has already faced the similar issue and with what approach they were able to implement this without violating the Rest API conventions.
This request:
POST /stores/:storeid/products/
can expose the product_id as part of the URI returned:
Location: /stores/{storeid}/products/{productid}
It does not need to tell you the technical id of the mapping row in your database. Remember, you don't have to expose database rows one-to-one in your REST API.
As a sidenote, if your clients are Hypertext driven (as they should be), they don't have to parse or construct URIs manually, therefore it would be not that important what you expose in the URIs.

Loopback: How to isolate a users queries to a specific datasource?

Let's say I have two users, A and B, with IDs 1 and 2 (respectively). Further, let's assume I have two datasources configured: X and Y.
How could I isolate ALL queries issued by user A to datasource X, and all by B to Y for some given remote method? For example, say that A wants to run 'find' for some model via the API - how could I make sure that the only results A will get are those which are accessible through datasource X?
I'm not sure I entirely understand why you would decide a datasource based on the current user, but in any case, I'm not sure you can do that with LoopBack ... at least, not easily. LoopBack is a model-driven framework - everything derives from the model. As such, all API endpoints go through a model (although you can set up custom routes). And each model is connected to a single datasource.
So, if I hit /api/Widget/13 there is no way to make that findById() call switch between two datasources, it will always hit whatever datasource the model is connected to.
Okay, that all said, the solutions I see are to:
Create a "dispatcher" and have that model do the appropriate thing.
Create a custom remote method on your existing model and do the decision making there and the find, etc on the correct datasource.
In either case, it's not straightforward, and not built-in. FYI, if you need to get the datasource you can access it from the LoopBack application object: MyModel.app.datasources.ds1