We have a web service API, where we will receive an input parameter value say for eg: X, Y, Z (Its a mandatory field in API). Also we have a GUI to configure throttle limit for each value (X = 100, Y = 50, Z = 30). Also this web service API is running on multiple nodes. Using an internal loadbalancer(LVS) the load is getting shared across multiple nodes. Now when we are receiving the request we have to throttle the incoming request using the limit configured. Please let me know any better suggestion or approach to proceed further on this.
We are using AXIS 2 framework for web service API.
Related
We have a webpage that queries an item from an API gateway which in turn calls a service that calls another service and so on.
Webpage --> API Gateway --> service#1 --> service#2 --> data store (RDMS, S3, Azure blob)
We want to make the operation resilient so we added a retry mechanism at every layer.
Webpage --retry--> API Gateway --retry--> service#1 --retry--> service#2 --retry--> data store.
This however could case a cascading failure because if the data store doesn't response on time, it will cause every layer to timeout and retry. In other words, if each layer has the same connection timeout and is configured to retry 3 times, then there will be a total of 81 retries to the data store (which is called a retry storm).
One way to fix this is to increase the timeout at each layer in order to give the layer below time to retry.
Webpage --5m timeout--> API Gateway --2m timeout--> service#1
This however is unacceptable because the timeout at the webpage will be too long.
How should I address this problem?
Should there only be one layer that retries? Which layer? And how can the layer know if the error is transient?
A couple possible solutions (and you can/should use both) would be to retry on different conditions and implement rate limiters/circuit breakers.
Retry On is a technique where you don't retry on every condition, but only specific conditions. This could be a specific error code or a specific header value. E.g. in your current situation, DO NOT retry on timeouts; only retry on server failures. In addition, you could have each layer retry on different conditions
Rate limiting would be to stick either a local or global rate limiter service inline to the connections. This would just help to short-circuit the thundering herd in the case that it starts up. E.g. rate limit the data layer to X req/s (insert real values here) and the gateway to Y req/s and then even if a service attempts lots of retries it won't pass too far down the chain. Similarly to this is circuit breaking, where each layer only permits X active connections to any downstream, so just another way to slow those retry storms.
Kubernetes and Istio already installed in the cluster. Three micro services deployed as PODs. The flow is
Micro service A to Micro Service B calls => HTTP
Micro service B to Micro service C calls => via Kafka
Micro service A expose a HTTP API to outside
I guess when client hit the Ingres, Istio generate traceId and spanId in HTTP header and enter to Service A.
Are these spanId and traceId propagate to Micro service B and C without using separate API like Spring Cloud sleuth?
No, Istio does not provide tracing headers propagation. However it can be configured on application side without use of 3rd party APIs.
According to Istio documentation:
Istio leverages Envoy’s distributed tracing feature to provide tracing integration out of the box. Specifically, Istio provides options to install various tracing backend and configure proxies to send trace spans to them automatically. See Zipkin, Jaeger and LightStep task docs about how Istio works with those tracing systems.
Istio documentation also has an example of application side header propagation for the bookinfo demo application:
Trace context propagation
Although Istio proxies are able to automatically send spans, they need some hints to tie together the entire trace. Applications need to propagate the appropriate HTTP headers so that when the proxies send span information, the spans can be correlated correctly into a single trace.
To do this, an application needs to collect and propagate the following headers from the incoming request to any outgoing requests:
x-request-id
x-b3-traceid
x-b3-spanid
x-b3-parentspanid
x-b3-sampled
x-b3-flags
x-ot-span-context
Additionally, tracing integrations based on OpenCensus (e.g. Stackdriver) propagate the following headers:
x-cloud-trace-context
traceparent
grpc-trace-bin
If you look at the sample Python productpage service, for example, you see that the application extracts the required headers from an HTTP request using OpenTracing libraries:
def getForwardHeaders(request):
headers = {}
# x-b3-*** headers can be populated using the opentracing span
span = get_current_span()
carrier = {}
tracer.inject(
span_context=span.context,
format=Format.HTTP_HEADERS,
carrier=carrier)
headers.update(carrier)
# ...
incoming_headers = ['x-request-id']
# ...
for ihdr in incoming_headers:
val = request.headers.get(ihdr)
if val is not None:
headers[ihdr] = val
return headers
The reviews application (Java) does something similar:
#GET
#Path("/reviews/{productId}")
public Response bookReviewsById(#PathParam("productId") int productId,
#HeaderParam("end-user") String user,
#HeaderParam("x-request-id") String xreq,
#HeaderParam("x-b3-traceid") String xtraceid,
#HeaderParam("x-b3-spanid") String xspanid,
#HeaderParam("x-b3-parentspanid") String xparentspanid,
#HeaderParam("x-b3-sampled") String xsampled,
#HeaderParam("x-b3-flags") String xflags,
#HeaderParam("x-ot-span-context") String xotspan) {
if (ratings_enabled) {
JsonObject ratingsResponse = getRatings(Integer.toString(productId), user, xreq, xtraceid, xspanid, xparentspanid, xsampled, xflags, xotspan);
When you make downstream calls in your applications, make sure to include these headers.
we are moving from monolithic to microservice architecture application, we're still in planning phase and we want to know what is the best practices of building it.
suppose we have two services :
User
Device
getUserDevices(UserId)
addDevice(DeviceInfo, UserId)
...
Each user has multiple devices
what is the most common, cleaner and proper way of asking the server to get all user devices ?
1- {api-url}/User/{UserId}/devices
needs another HTTP request to communicate with Device service.
for user X, get linked devices from User service.
// OR
2- {api-url}/Device/{UserId}/devices
for user X, get linked devices from Device service.
There are a lot of classic patterns available to solve such problems in Microservices. You have 2 microservices - 1 for User (Microservice A) and 1 for Device (Microservice B). The fundamental principle of a microservice is to have a separate database for each of the microservice. If any microservice wants to talk to each other (or to get data from another microservice), they can but they would do it using an API. Another way for communication between 2 microservices is by events. When something happens in Microservice A, it will raise an event and push it to a central event store or a message queue and Microservice B would subscribe to some or all of the events emitted by A.
I guess in your domain, A would have methods like - Add/Update/Delete a User and B would have Add/Update/Delete a device. Each user can have its own unique id and other data fields like Name, Address, Email etc. Each device can have its own unique id, a user id, and other data fields like Name, Type, Manufacturer, Price etc. Whenever you "Add" a device, you can send a POST request or a command (if you use CQRS) to Device Microservice with the request containing data about device + user-id and it could raise an event called "DeviceAdded". It can also have events corresponding to Update and Delete like "DeviceUpdated" and "DeviceRemoved". The microservice A can subscribe to events - "DeviceAdded", "DeviceRemoved", and "DeviceUpdated" events emitted by B and whenever any such event is raised, it will handle that event and denormalize that event into its own little database of Devices (Which you can call UserRelationships). In future, it can listen to events from other microservices too (so your pattern here would be extensible and scalable).
So now to get all devices owned by a user, all you have to do is make an end-point in User Microservice like "http://{microservice-A-host}:{port}/user/{user-id}/devices" and it will return you a list of the devices by querying for user-id in its own little database of UserRelationships which you must have been maintaining through events.
Good Reference is here: https://www.nginx.com/blog/event-driven-data-management-microservices/
it may really be either way, but to my liking, I would choose to put it under /Devices/{userId}/devices as you are looking for the devices given the user id. I hope this helps. Have a nice one!
You are requesting a resource from a service, resource being a device and service being a device service.
From a rest standpoint, you are looking for a resource and your service is providing various methods to manipulate that resource.
The following url can be used.
[GET] ../device?user_id=xyz
And device information can be fetched via ../device/{device_id}
Having said that, if you had one service that is providing for both user and device data than the following would have made sense.
[GET] ../user/{userId}/device
Do note that this is just a naming convention and you can pick what suits best for you, thing is pick one and hold onto it.
When exposing the api consistency is more important.
One core principle of the microservice architecture is
defining clear boundaries and responsibilities of each microservice.
I can say that it's the same Single Responsibility Principle from SOLID, but on macro level.
Сonsidering this principle we get:
Users service is responsible for user management/operations
Devices service is responsible for operations with devices
You question is
..proper way of asking the server to get all user devices
It's 100% responsibility of the Devices service and Users service nothing know about devices.
As I can see you thinking only in routing terms (yes API consistency is also important).
From one side the better and more logical URL is /api/users/{userId}/devices
- you try to get user's devices, these devices belong to user.
From other side you can use the routes like /api/devices/user/{userId} (/api/devices/{deviceId}) and that can be more easily processed
by the routing system to send a request to the Devices service.
Taking into account other constraints you can choose the option that is right for your design.
And also small addition to:
needs another HTTP request to communicate with Device service.
in the architecture of your solution you can create an additional special and separate component that routes the requests to the desired microservice, not only direct calls are possible from one microservice to another.
You should query the device service only.
And treat the user id like a filter in the device service. For eg: you should search on userid similar to how you would search device based on device type. Just another filter
Eg : /devices?userid=
Also you could cache some basic information of user in device service, to save round trips on getting user data
With microservices there is nothing wrong with both the options. However the device api makes more sense and further I'll prefer
GET ../device/{userId}/devices
over
GET ../device?user_id=123
There are two reasons:
As userId should already be there with devices service you'll save one call to user service. Otherwise it'll go like Requester -> User service -> Device Service
You can use POST ../device/{userId}/devices to create new device for particular user. Which looks more restful then parameterized URL.
I have a proxy service, but according to a parameter in the request I need to invoke a different business service (e.g. if the parameter is 1 I need to invoke service A otherwise Service B), and each service has a different response, so for service A I need to transform the response but for Service B it's not necessary!
I don't know how to do this in a PS, and what is the best way using Routing or Service Call out, having into account that is for a system that will have a lot transactions. What I have in the picture is the scenario por Service 1, I need to include both scenarios
I think what you can do in this case is use a Routing Table, which you would be able to set a variable somewhere earlier in your pipeline to determine which route you'll take in this table through use of an XQuery expression.
Let's say my web service is located at http://localhost:8080/foo/mywebservice and my WSDL is at http://localhost:8080/foo/mywebservice?wsdl.
Is http://localhost:8080/foo/mywebservice an endpoint, i.e., is it the same as the URI of my web service or where the SOAP messages received and unmarshalled?
Could you please explain to me what it is and what the purpose of it is?
This is a shorter and hopefully clearer answer...
Yes, the endpoint is the URL where your service can be accessed by a client application. The same web service can have multiple endpoints, for example in order to make it available using different protocols.
Updated answer, from Peter in comments :
This is de "old terminology", use directally the WSDL2 "endepoint"
definition (WSDL2 translated "port" to "endpoint").
Maybe you find an answer in this document : http://www.w3.org/TR/wsdl.html
A WSDL document defines services as collections of network endpoints, or ports. In WSDL, the abstract definition of endpoints and messages is separated from their concrete network deployment or data format bindings. This allows the reuse of abstract definitions: messages, which are abstract descriptions of the data being exchanged, and port types which are abstract collections of operations. The concrete protocol and data format specifications for a particular port type constitutes a reusable binding. A port is defined by associating a network address with a reusable binding, and a collection of ports define a service. Hence, a WSDL document uses the following elements in the definition of network services:
Types– a container for data type definitions using some type system (such as XSD).
Message– an abstract, typed definition of the data being communicated.
Operation– an abstract description of an action supported by the service.
Port Type–an abstract set of operations supported by one or more endpoints.
Binding– a concrete protocol and data format specification for a particular port type.
Port– a single endpoint defined as a combination of a binding and a network address.
Service– a collection of related endpoints.
http://www.ehow.com/info_12212371_definition-service-endpoint.html
The endpoint is a connection point where HTML files or active server pages are exposed. Endpoints provide information needed to address a Web service endpoint. The endpoint provides a reference or specification that is used to define a group or family of message addressing properties and give end-to-end message characteristics, such as references for the source and destination of endpoints, and the identity of messages to allow for uniform addressing of "independent" messages. The endpoint can be a PC, PDA, or point-of-sale terminal.
A web service endpoint is the URL that another program would use to communicate with your program. To see the WSDL you add ?wsdl to the web service endpoint URL.
Web services are for program-to-program interaction, while web pages are for program-to-human interaction.
So:
Endpoint is: http://www.blah.com/myproject/webservice/webmethod
Therefore,
WSDL is: http://www.blah.com/myproject/webservice/webmethod?wsdl
To expand further on the elements of a WSDL, I always find it helpful to compare them to code:
A WSDL has 2 portions (physical & abstract).
Physical Portion:
Definitions - variables - ex: myVar, x, y, etc.
Types - data types - ex: int, double, String, myObjectType
Operations - methods/functions - ex: myMethod(), myFunction(), etc.
Messages - method/function input parameters & return types
ex: public myObjectType myMethod(String myVar)
Porttypes - classes (i.e. they are a container for operations) - ex: MyClass{}, etc.
Abstract Portion:
Binding - these connect to the porttypes and define the chosen protocol for communicating with this web service.
- a protocol is a form of communication (so text/SMS, vs. phone vs. email, etc.).
Service - this lists the address where another program can find your web service (i.e. your endpoint).
In past projects I worked on, the endpoint was a relative property. That is to say it may or may not have been appended to, but it always contained the protocol://host:port/partOfThePath.
If the service being called had a dynamic part to it, for example a ?param=dynamicValue, then that part would get added to the endpoint. But many times the endpoint could be used as is without having to be amended.
Whats important to understand is what an endpoint is not and how it helps. For example an alternative way to pass the information stored in an endpoint would be to store the different parts of the endpoint in separate properties. For example:
hostForServiceA=someIp
portForServiceA=8080
pathForServiceA=/some/service/path
hostForServiceB=someIp
portForServiceB=8080
pathForServiceB=/some/service/path
Or if the same host and port across multiple services:
host=someIp
port=8080
pathForServiceA=/some/service/path
pathForServiceB=/some/service/path
In those cases the full URL would need to be constructed in your code as such:
String url = "http://" + host + ":" + port + pathForServiceA + "?" + dynamicParam + "=" + dynamicValue;
In contract this can be stored as an endpoint as such
serviceAEndpoint=http://host:port/some/service/path?dynamicParam=
And yes many times we stored the endpoint up to and including the '='. This lead to code like this:
String url = serviceAEndpoint + dynamicValue;
Hope that sheds some light.
Simply put, an endpoint is one end of a communication channel. When an API interacts with another system, the touch-points of this communication are considered endpoints. For APIs, an endpoint can include a URL of a server or service. Each endpoint is the location from which APIs can access the resources they need to carry out their function.
APIs work using ‘requests’ and ‘responses.’ When an API requests information from a web application or web server, it will receive a response. The place that APIs send requests and where the resource lives, is called an endpoint.
Reference:
https://smartbear.com/learn/performance-monitoring/api-endpoints/
An Endpoint is specified as a relative or absolute url that usually results in a response. That response is usually the result of a server-side process that, could, for instance, produce a JSON string. That string can then be consumed by the application that made the call to the endpoint. So, in general endpoints are predefined access points, used within TCP/IP networks to initiate a process and/or return a response. Endpoints could contain parameters passed within the URL, as key value pairs, multiple key value pairs are separated by an ampersand, allowing the endpoint to call, for example, an update/insert process; so endpoints don’t always need to return a response, but a response is always useful, even if it is just to indicate the success or failure of an operation.
A endpoint is a URL for web service.And Endpoints also is a distributed API.
The Simple Object Access Protocol (SOAP) endpoint is a URL. It identifies the location on the built-in HTTP service where the web services listener listens for incoming requests.
Reference: https://www.ibm.com/support/knowledgecenter/SSSHYH_7.1.0.4/com.ibm.netcoolimpact.doc/dsa/imdsa_web_netcool_impact_soap_endpoint_c.html