Our application integration flow is defined as splitter -> ws gateway -> aggregator
The splitter splits request into a list of account numbers; so that for each account number a web service call is initiated and the responses from multiple web service calls are aggregated in the aggregator.The channel between splitter and ws gateway is defined with dispatcher "commonj WorkManagerTaskExecutor" so that each webservice call is initiated parallelly in different threads.
If at least some of the web service call responds properly; even if all other calls result in SoapFault; we need to handle the scenario by using the data from the successful responses with a warning message quoting the error message from the fault response.
The issue is that resolveFault() method of FaultMessageResolver defined in the ws gateway does not return anything and the control never reaches the aggregator if at least one of the parallel web service call fails. Is there any way to handle such a scenario.
You can inject SoapFaultMessageResolver to the <int-ws:outbound-gateway> (fault-message-resolver). This one has pretty simple code:
public void resolveFault(WebServiceMessage message) throws IOException {
SoapMessage soapMessage = (SoapMessage) message;
throw new SoapFaultClientException(soapMessage);
}
So, you failed WS invocation will end up with an Exception.
Add <int-ws:request-handler-advice-chain> to your <int-ws:outbound-gateway> and place there an instance of ExpressionEvaluatingRequestHandlerAdvice. specify its errorChannel and do some agnostic logic in that sub-flow and send some specific message to your aggregator. Don't forget to carry sequenceDetails headers with that messages.
Having all messages in group aggregator will be able to release is as normal one.
In the end you can analyze result List for errors and normal responses.
Related
Is it possible to invoke a AWS Step function by API Gateway endpoint and listen for the response (Until the workflow completes and return the results from end step)?
Currently I was able to find from the documentation that step functions are asynchronous by nature and has a final callback at the end. I have the need for the API invocation response getting the end results from step function flow without polling.
I guess that's not possible.
It's async and also there's the API Gateway Timeout
You don't need get the results by polling, you can combine Lambda, Step Functions, SNS and Websockets to get your results real time.
If you want to push a notification to a client (web browser) and you don't want to manage your own infra structure (scaling socket servers and etc) you could use AWS IOT. This tutorial may help you to get started:
http://gettechtalent.com/blog/tutorial-real-time-frontend-updates-with-react-serverless-and-websockets-on-aws-iot.html
If you only need to send the result to a backend (a web service endpoint for example), SNS should be fine.
This will probably work: create an HTTP "gateway" server that dispatches requests to your Steps workflow, then holds onto the request object until it receives a notification that allows it to send a response.
The gateway server will need to add a correlation ID to the payload, and the step workflow will need to carry that through.
One plausible way to receive the notification is with SQS.
Some psuedocode that's vaguely Node/Express flavoured:
const cache = new Cache(); // pick your favourite cache library
const gatewayId = guid(); // this lets us scale horizontally
const subscription = subscribeToQueue({
filter: { gatewayId },
topic: topicName,
});
httpServer.post( (req, res) => {
const correlationId = guid();
cache.add(correlationId, res);
submitToStepWorkflow(gatewayId, correlationId, req);
});
subscription.onNewMessage( message => {
const req = cache.pop(message.attributes.correlationId);
req.send(extractResponse(message));
req.end();
});
(The hypothetical queue reading API here is completely unlike aws-sdk's SQS API, but you get the idea)
So at the end of your step workflow, you just need to publish a message to SQS (perhaps via SNS) ensuring that the correlationId and gatewayId are preserved.
To handle failure, and avoid the cache filling with orphaned request objects, you'd probably want to set an expiry time on the cache, and handle expiry events:
cache.onExpiry( (key, req) => {
req.status(502);
req.send(gatewayTimeoutMessage());
req.end();
}
This whole approach only makes sense for workflows that you expect to normally complete in the sort of times that fit in a browser and proxy timeouts, of course.
I have a service which accepts HTTP requests from a customer site. The service then sends an HTTP request to a transactional email provider with information provided in the initial request to the service. The workflow looks like this:
CustomerSite ⟷ EmailService ⟷ TransactionEmailProvider
I can think of two possibilities for handling requests so that errors from the TransactionalEmailProvider can be reported to the CustomerSite.
The EmailService immediately sends an asynchronous request to
TransactionalEmailProvider when it receives a request from a
CustomerSite. The EmailService immediately responds to the
CustomerSite with a success code if the request was properly
formed. If a failure happened when sending a request to the
TransactionalEmailProvider, the EmailService sends a failure
notification using a POST request back to the EmailService using a
webhook implementation.
The EmailService sends a request to the TransactionalEmailProvider, and awaits a response before responding to the CustomerSite request with either a success or a failure.
Right now I'm implementing the first version because I don't want the responsiveness of the EmailService to be dependent on the responsiveness of the TransactionalEmailProvider.
Is this a reasonable way to process HTTP requests that are dependent upon a second level of HTTP requests? Are there situations in which one would be preferred over the other?
Is this a reasonable way to process HTTP requests that are dependent upon a second level of HTTP requests? Are there situations in which one would be preferred over the other?
It really depends on the system requirements, it depends on how you want to behave in case of failure of some of its components or under varying workload.
If you want your system to be reactive or scalable you should use asynchronous requests whenever possible. For this your system should be message driven. You could read more about reactive system here. This seems like your first option.
If you want a simpler system then use synchronous/blocking requests, like your option no. 2
I have a function to give recommendations to users. This function need to make a lot of calcs to start, but after start it use the already calculed matrix on memory. After this, any other calc that is made, "fills" the object in memory to continuous learning.
My intention is to use this function to website users, but the response need to come from the same "object" in memory and need to be sequential by request because it is not thread safe.
How is the best way to get this working? My first idea was use signalr so the user dont need to wait to response and a queue to send the requests to objects. But how the signalr can receive the response for this specific request?
The entire flow is:
User enter on a page.
A javascript will call a service with the user ID and actual page.
The server will queue the ID an page.
The service will be calculating the results for each request on queue and sending responses.
The server will "receive" the response and send back to client.
The main problem is that I dont see a way to the service receive the response to send back to client until it is complete, without need to be looping in queues.
Thanks!
If you are going to use SignalR, I would suggest using a hub method to accept these potentially long running requests from the client. By doing so it should be obvious "how the signalr can receive the response for this specific request".
You should be able to queue your calculations from inside your hub method where you will have access to the caller's connection id (via the Context.ConnectionId property).
If you can await the results of your queued operation inside of the hub method you queue from, you can then simply return the result from your hub method and SignalR will flow the result back to the calling JavaScript. You can also use Clients.Caller.... to send the result back.
If you go this route I suggest you use async/await instead of blocking request threads waiting for your long-running calculations to complete.
http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server
If you can't process your calculation results from the same method you queued the calculation from, you still have options. Just be sure to queue the caller's connection id and a request id along with the calculation to be processed.
Then, you can process the results of all your calculations from outside of your hub using GlobalHost.ConnectionManager.GetHubContext:
private IHubContext _context = GlobalHost.ConnectionManager.GetHubContext<MyHub>()
// Call ProcessResults whenever results are ready to send back to the client
public void ProcessResults(string connectionId, uint requestId, MyResult result)
{
// Presumably there's JS code mapping request id's to results
// if you can have multiple ongoing requests per client
_context.Clients.Client(connectionId).receiveResult(requestId, result);
}
http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server#callfromoutsidehub
I am implementing a Spring MessageListener that is listening to a JMS Queue to process messages containing XML.
My bean ProposalSOAListener will be processing about 5 or more XML messages from the queue. My code is below.
Is there a way to specify different methods on this class to handle different XML messages?
public class ProposalSOAListener implements MessageListener {
public void onMessage(Message message) {
if (message instanceof TextMessage) {
try {
System.out.println(((TextMessage) message).getText());
} catch (JMSException ex) {
throw new RuntimeException(ex);
}
}
else {
throw new IllegalArgumentException("Message must be of type TextMessage");
}
}
} // end of ProposalSOAListener class
There's a bunch of architectural questions begged by your question. Do you want this mesasge listener to do the work, or hand it off to another component? Are there transactional considerations at play? Do you have memory constraints - i.e. do you want streaming based XML processing or not? Do
The good news is that you have a lot of the pieces to this puzzle available to you within Spring.
A simple next step would be to use Spring Object XML Marshalling (OXM), choose one of the techniques, and wire the marshaller into your listener bean.
See http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/oxm.html
Another technique would be to use the Spring ApplicationEvent interface, read the messages coming in off the queue and publish them internally to listeners of the specific types. That could be used in combination with the above object marshalling.
Last but not least, if this is SOAP web services - you can take a look at Spring WS, it uses the similar message containers to pull messages off the wire, marshall them, and invoke a spring ws endpoint (ie. the service interface that satisfies that interface contract).
http://static.springsource.org/spring-ws/sites/2.0/reference/html/server.html#d4e907
Spring Integration project is highly recommended for this kind of a problem. Essentially you will have to implement a jms inbound gateway to get your message in. You can then transform this to an object at this point, then route the message to the appropriate service-activator component, which can map to your instance and method.
I have a legacy VB6 application that needs to make asynchronous calls to a web service. The web service provides a search method allows end-users to query a central database and view the results from within the application. I'm using the MSXML2.XMLHTTP to make the requests, and have written a SearchWebService class that encapsulates the web service call and code to handle the response asychronously.
Currently, the SearchWebService raises one of two events to the caller: SearchCompleted and SearchFailed. A SearchCompleted event is raised that contains the search results in a parameter to the event if the call completes successfully. A SearchFailed is raised when any type of failure is detected, which can be anything from an improperly-formatted URL (this is possible because the URL is user-configurable), to low-level network errors such as "Host not found", to HTTP errors such as internal server errors. It returns a error message string to the end-user (which is extracted from the web service response body, if present, or from the HTTP status code text if the response has no body, or translated from the network error code if a network error occurs).
Because of various security requirements, the calling application does not access the web service directly, but instead accesses it through a proxy web server running at the customer site, which in turn accesses the actual web service through via a VPN. However, the SearchWebService doesn't know that the calling application is accessing the web service through a proxy: it's just given a URL and told to make the request. The existence of the proxy is a application-level requirement.
The problem is that from an end-user perspective, it's important that the calling application be able to distinguish between low-level network errors versus HTTP errors from the web service, and to distinguish proxy errors from remote web server errors. For example, the application needs to know if a request failed because the proxy server is down, or because the remote web service that the proxy is accessing is down. An application-specific message needs to be presented to the end-user in each case, such as "Search web service proxy server appears to be down. The proxy server may need to be restarted" versus "The proxy is currently running but the remote web server appears to be unavailable. Please contact (name of person in charge of the remote web server)." I could handle this directly in the SearchWebService class, but it seems wrong to generate these application-specific error messages from such a generic class (and the class might be used in environments that don't require a proxy, where the error messages would no longer make sense).
This distinction is important for troubleshooting: a proxy server problem can usually be resolved by the customer, but a remote web server error has to handled by a third party.
I was thinking one way to handle this would be to have the SearchWebService class detect different types of errors and raise different events in each case. For example, instead of a single SearchFailed event, I could have a NetworkError event for low-level network errors (which would indicate a problem accessing the proxy server), a ConfigurationError event for invalid properties on the SearchWebService class (such as passing an improperly-formatted URL), and a ServiceError for errors that occur on the remote web server (implying that the proxy is working properly but the remote server returned an error).
Now that I think about it, there is also an additional error scenario: it could be possible that the proxy server is running properly, but the remote web server is down, or the proxy server has been misconfigured.
Is the approach of using multiple error events to classify different classes of error a reasonable solution to this problem? For the last scenario (the proxy is running but the remote server cannot be reached), I'm guessing I may have to set up the proxy to return a specific HTTP error code so that client can detect this situation (i.e. something more specific than a 500 response).
Originally I kept the single SearchFailed event and simply added an additional errorCode parameter to the event, but that got messy quickly, especially in cases where there wasn't a logical error code to use (such as if the VB6 raises a "real" error, i.e. if the XMLHTTP class isn't registered).
I think that some ideas I've used with Java exceptions may apply here.
Having a large number of different Exceptions gets pretty messy, yet we need to give enough detail to the user so we don't want to lose information.
Hence I have a small number of specific Exceptions, which I guess would correspond to your Events:
InvalidRequestEvent: Used when the user specifies bad information
TransientErrorEvent: used when there's infrastructure issues when a retry might work.
I tend to work in environments where we have clusters of servers so if a user request hits a dying server then if he resubmits he'll probably get a good one, hence from his perspective a simple retry often works. However sometimes the error is with a service such as the Network or Database and in which case the user needs diagnostic information to report to the helpdesk. Hence we need to decide on the extra information to put into the exception. This is (if I understand you correctly) your question.
In the case of InvalidRequestException we would bet giving some information about the problems with the input. It could be on the lines of "Mismatched parenthese" or "Unknown column CUTSOMER in table ORDER". In the case of TransientErrorException it could be "Proxy server is down".
Now depending upon your exact requirments you may not actually choose to put that text in the Exception, but rather an error number which the presentation layer converts to a locale-specific string (English, French ...).
So either Exception might contain something like this (sorry for that Java syntax, but I hope the idea is clear):
BaseException {
String ErrorText; // the error text itself
// OR if you want to allow for internationaliation
int ErrorCode; // my application specific code, corresponds to text held by the UI
String[] params; // specific parameters to be substitued in the error text
// CUTSOMER and ORDER in my example above
int SystemErrorCode; // If you have an underlying error code it goes here
String SystemErrorText; // any further diagnoistic you might need to give to
// the user so that they can report the problem to the
// help desk.
// OR instead of the text (this is something I've seen done)
int SystemErrorTag; // A unique id for this particular error problem.
// This server systems will label their message in the
// server logs. Users just tell the help desk this number
// they don't need to read detailed server error text.
}