I am learning about microservices and I get very confused when it comes to data duplication. From different sources on the internet I get very different opinions on the subject, but the main conclusion that I got is that it depends if it's good on your use case or application.
Now I am struggling because I am trying to make a very simple microservice where I think data duplication is not necessary, but can't seem to think of a good alternative to it.
My Application
I am basically making a Twitter clone to understand the basics of microservices using AWS. To start I have 3 services:
Timeline Service
Post (or tweet) Service
Follow Service
Timeline service
The timeline service should return the timeline of the user that asked it. It should check who the user is, that requests his/her timeline is following, get the 100 most recent posts from those users and return it to the user (that requested his/her timeline)
Posts/tweet Service
When a user is posting a new tweet/post, the post timeline should save the post in the post database to keep track of who posts what.
Follow Service
When a user wants to follow another user the follow service should keep track of that.
If I put it into an image it would look like this:
The problem
My question is about the timeline service. Should the timeline
Have two (in memory) databases that also keep track of who follows who and who posts which tweets to give the user quickly their timeline back
Or should the timeline service call the different services to get that information?
The 'bad' thing from the last option would be that if my posts service is failing and can't send or receive requests, my timeline service also does not work and you create dependency which goes against microservice principles.
Since the last option seems the worst one to me, I would say data duplication is the better option because of response time and to keep the different services from relying on each other.
Can anyone help me to get on the right track with this problem?
We are working in the office using Oracle APEX to create some web services. After beating our heads against the wall we finally figured out that the terms get & post when talking about RESTful services are not the same as standard HTTP.
We will need to create two types of web services;
In the first, the external application passes a few fields and the method queries a record and sends the data back to the external application.
In the second an external application builds a 30 field record that needs to be inserted by the method into the DB.
In a normal web app we would in both cases typically http post the data to to a cgi (asp, aspx, pl...) page that would parse the fields and do what needs to be done.
Since we have written the new app in APEX, and we would like to join the late 20th century, we thought we would use a web service on the provider side. The consumer will be AS3/flex/flash.
Can we still use the traditional http post and/or http get methodology of defining name=data pairs and send them to the web service?
What we have seen so far is that the data passed (at least in an APEX Restful GET) needs to be positional and not name=data pairs.
I guess the first question is where can we get some basic concepts on how to pass data using "RESTful" get & post? Something aimed at a web service dolt would be good.
Another question is if the HTTP get/post methodolgy can be used, how would it be done in Oracle APEX v5.0?
There is a lot of questions inside your post, perhaps if you narrow it down to a specific problem, you'd get better answers. I've used RESTful services with APEX 4.2, 5, 5.1 and they work just fine.
In the first, the external application passes a few fields and the method queries a record and sends the data back to the external application.
This is typical GET request. You pass the parameters (I usually pass them as part of the URL), use them in query, and return the values. You can even pass them to PL/SQL block, do whatever processing you need, then return the results.
In the second an external application builds a 30 field record that needs to be inserted by the method into the DB.
This is typcial POST request (although you still can use GET but not recommended). The format of how the data is being sent is important. I've done that using JSON format, and received as bind variable :body at APEX end. In APEX you can parse it and insert it into a table.
Few resources might help you to get going:
http://www.oracle.com/webfolder/technetwork/tutorials/obe/db/apex/r50/Restful%20Services/restful_services.html#section1
This is especially helpful for POST:
https://ruepprich.wordpress.com/2016/03/22/apex-rest-post/
I am in the process of building a RESTful API for my application. There are very few services that are public and the rest require authentication and authorization.
To be clear, my question is NOT about authenticating web services. I have already decided to send an HTTP header with an access token provided by the server. The reasons for this include:
Creating a "session" that can track the user activity
Timeout access tokens after XXX amount of inactivity
Track user behavior patterns for each "session"
So far, this approach is working fine. I am interested in any design guidelines for providing a "Login" service. I don't want to just authenticate a request, but I want to track usage of the web service against a "session".
In addition to "session" tracking, we have requirements that require that we track failed login attempts and take appropriate action after XXX number of failed attempts as well as password expiring and email address verification before authorizing, etc.
Specifically, I am concerned with the best way to design the URI's for this. One option would be:
/api/authentication/login?username=UN&password=PW
That could return the access token to be used in the header for secure service calls. Is this a good approach? Is there a better approach? Is there a better patter to use for naming the URI?
My biggest problem is that the URI is not purely sticking with the "URI's should represent resources" approach. End the end it is probably not a big deal, but I am wondering if there are better ways.
Thanks!
Often, RESTful APIs like to be stateless. That means that the API itself doesn't care about keeping a session, and doesn't.
What you do is authenticate 1 time, and then get a temporary key. That key eventually is no good anymore because the key has information in it about when it will expire.
Also, since these large APIs are built on message queues, they know timestamps for each action. and they can basically keep track of activity.
So, in RESTful API design, you often have scenarios where your URL has resources in it, and then there are all sorts of additional things that need to be set.
A good rule of thumb is to hide the complexity behind your ?. A typical use case of this philosophy is where you have a bunch of filter options on a get request of /some/resource. How is this relevant? Well, if you remember that its not a mortal sin to decorate your resource based API with other stuff occassionally, then you can treat other scenarios similarly when you feel like resourcefulness may be in question, but essentially you still have RPC-ish endpoints that need to exist to make your API fully functional for your needs. Or, of course, you can just arbitrarily set certain HTTP verbs to equal those things.
If you want to extend your resources with additional functionality, try to stick to the resource structure in your base url of the call, and then decorate it with your one-off needs.
Resource: /api/authentication
With modifier: /api/authentication/login
With data: /api/authentication/login?username=UN&password=PW
Its not so bad. But again, if you wanted to go completely restful, you could say something like this (this is pure conjecture, you need to decide these things for yourself):
Get logged in status - GET - /api/authentication/:id
Create / Update logged in status - POST / PUT - /api/authentication(/:id)
Log out - DELETE - /api/authentication/:id
... or you could have omitted the :id route and just hid that information in the body of data appended to the call, aka hiding complexity
I have a web service for a forum, and I have the following two resources:
/threads/frontpage, returns all the posts that belong on the front page of my site
/users/1, users/2, etc. returns users specified by the number.
And I want to make it so the user can ask for "all the threads on the frontpage, plus all the users that posted to those threads".
Normally I would separate it out into multiple requests, but that takes more time.
My only thought is to make a URL like this:
/threads/frontpage?includePostingUsers=true
In a way, it's taking the original result, and then using it to include additional info, in this case, the users that posted the threads.
But that doesn't seem very RESTful. There's a verb in there, and it just feels awkward. How would I do this RESTfully?
Thanks!
Having a verb in a query parameter is not necessary un-RESTful. I would say that it is common (and more important) to have clean resources and verbs, but to include more involved functionality into query parameters, as you would do with ?includePostingUsers=true.
Apigee published a nice pragmatic booklet on Crafting Restful APIs:
Web API Design – Crafting Interfaces that Developers Love (PDF), and a related video with real world observations: https://www.youtube.com/watch?v=QpAhXa12xvU#t=39m
If you don't want to show ?includePostingUsers=true in url then use Post request.
And why don't you just pass parameter like
/threads/frontpage?uid=1
Looking for clear and concise explanations of this concept.
A RESTful application is an application that exposes its state and functionality as a set of resources that the clients can manipulate and conforms to a certain set of principles:
All resources are uniquely addressable, usually through URIs; other addressing can also be used, though.
All resources can be manipulated through a constrained set of well-known actions, usually CRUD (create, read, update, delete), represented most often through the HTTP's POST, GET, PUT and DELETE; it can be a different set or a subset though - for example, some implementations limit that set to read and modify only (GET and PUT) for example
The data for all resources is transferred through any of a constrained number of well-known representations, usually HTML, XML or JSON;
The communication between the client and the application is performed over a stateless protocol that allows for multiple layered intermediaries that can reroute and cache the requests and response packets transparently for the client and the application.
The Wikipedia article pointed by Tim Scott gives more details about the origin of REST, detailed principles, examples and so on.
The best explanation I found is in this REST tutorial.
REST by way of an example:
POST /user
fname=John&lname=Doe&age=25
The server responds:
200 OK
Location: /user/123
In the future, you can then retrieve the user information:
GET /user/123
The server responds:
200 OK
<fname>John</fname><lname>Doe</lname><age>25</age>
To update:
PUT /user/123
fname=Johnny
Frankly, the answer depends on context. REST and RESTful have meanings depending on what language or framework you're using or what you're trying to accomplish. Since you've tagged your question under "web services" I'll answer in the context of RESTful web services, which is still a broad category.
RESTful web services can mean anything from a strict REST interpretation, where all actions are done in a strict "RESTful" manner, to a protocol that is plain XML, meaning its not SOAP or XMLRPC. In the latter case, this is a misnomer: such a REST protocol is really a "plain old XML" (or "POX") protocol. While REST protocols usually use XML and as such are POX protocols, this doesn't necessarily have to be the case, and the inverse is not true (a just because a protocol uses XML doesn't make it RESTful).
Without further ado, a truly RESTful API consists of actions taken on objects, represented by the HTTP method used and the URL of that object. The actions are about the data and not about what the method does. For example, CRUD actions (create, read, update, and delete) can map to a certain set of URLs and actions. Lets say you are interacting with a photo API.
To create a photo, you'd send data via a POST request to /photos. It would let you know where the photo is via the Location header, e.g. /photos/12345
To view a photo, you'd use GET /photos/12345
To update a photo, you'd send data via a PUT request to /photos/12345.
To delete a photo, you'd use DELETE /photos/12345
To get a list of photos, you'd use GET /photos.
Other actions might be implemented, like the ability to copy photos via a COPY request.
In this way, the HTTP method you're using maps directly to the intent of your call, instead of sending the action you wish to take as part of the API. To contrast, a non-RESTful API might use many more URLs and only use the GET and POST actions. So, in this example, you might see:
To create a photo, send a POST to /photos/create
To view a photo, send a GET to /photos/view/12345
To update a photo, send a POST to /photos/update/12345
To delete a photo, send a GET to /photos/delete/12345
To get a list of photos, send a GET to /photos/list
You'll note how in this case the URLs are different and the methods are chosen only out of technical necessity: to send data, you must use a POST, while all other requests use GET.
Just a few points:
RESTFul doesn't depend on the framework you use. It depends on the architectural style it describes. If you don't follow the constraints, you're not RESTful. The constraints are defined in half a page of Chapter 5 of Roy Fielding's document, I encourage you to go and read it.
The identifier is opaque and does not cary any information beyond the identification of a resource. It's a nmae, not input data, just names. as far as the client is concerned, it has no logic or value beyond knowing how to build querystrings from a form tag. If your client builds its own URIs using a schema you've decided up-front, you're not restful.
The use or not use of all the http verbs is not really the constraint, and it's perfectly acceptable to design an architecture that only supports POST.
Caching, high decoupling, lack of session state and layered architecture are the points few talk about but the most important to the success of a RESTful architecture.
If you don't spend most of your time crafting your document format, you're probably not doing REST.
It means using names to identify both commands and parameters.
Instead of names being mere handles or monikers, the name itself contains information. Specifically, information about what is being requested, parameters for the request, etc..
Names are not "roots" but rather actions plus input data.
I've learned the most from reading the articles published on InfoQ.com:
http://www.infoq.com/rest and the RESTful Web Services book (http://oreilly.com/catalog/9780596529260/).
./alex
Disclaimer: I am associated with InfoQ.com, but this recommendation is based on my own learning experience.