Problem:
our system, running on AWS, uses an external service and we need to test specific flows/APIs of it. There is no way to completely mock this external service.
Desired solution:
we need an entity on the network which would monitor, catch specific messages from internal services and reply back in the name of the external service. I don't know if this is feasible at all. What i've found so far is that we probably need a transparent proxy. Maybe Squid...
Question:
can you recommend a solution(framework, tool, etc) that could do the above described tasks? If you can give details how to implement that would be great too.
Related
Is there a common way to establish a network connection from a CloudFoundry-Service to a CloudFoundry App which the service is bound to.
In typical fashion apps receive their bind credentials and establish network connections to provisioned service for example databases.
It would be very handy to establish a connection from a service to an app, so the service could scrape endpoints that are provided by the app.
Any thoughts on this, why is it / or isn't it possible, why could it be a bad idea.
Normally, you have your service and the application receives credentials from the service through the service binding (i.e. VCAP_SERVICES).
You want to reverse this arrangement, which is fine, but the service will need to have some way to know how to reach the applications. The way to do this would be through routes bound to your application.
I have seen something like this done before, this is roughly the process. I'm sure you can adapt it to your requirements.
Create a service broker. The broker is responsible for managing service instances and service credentials. The broker is notified when an instance is created and when a binding occurs. Your broker will need to handle these requests.
The broker, in addition to its normal responsibilities, is going to need to maintain state indicating which applications have instances & bindings. In addition, the broker is going to need to use the org/space/app guids it's provided through the service broker API and talk to the CloudFoundry API to fetch the routes for the applications that are bound to it. You don't usually get these through the service broker API, but since you want to talk to the applications from the service, you need this information. It gives the service a way to communicate with the application.
Your broker may also provide the service in question (i.e. talking to applications), or it can delegate to some other process/container/VM to provide the service. If your service does the latter, then you need a way to a.) create the process/container/VM and b.) pass along the information it requires to talk to your application.
Obviously, you need to code the logic that will take the routes for applications that have created instances and bindings and communicate with them.
There can be some limitations with using the routes. First, not all routes are public. For internal routes, it would be kind of complicated to allow the broker/service to talk to the app. The broker/service would need to be an application on CF and you would need to specifically allow that communication (would require more API calls). Second, some apps just don't have routes. Perhaps this won't happen in your case, but it's worth considering. Lastly, not all routes are HTTP, some can be TCP as well. Your broker/service would need to handle both of those.
A variation on the above process, instead of using routes or talking to the API, you could have your broker/service provide some mechanism through the credentials to the application such that it registers itself with the broker/service. Thus when your applications start, they'll read the service info, register with the service and then go about their business. In this way, the application would have some additional flexibility about what information it provides when it registers with the broker/service. The downside is that the app has to do some work to be compatible.
I want to know how can we send a message to a particular actor in cluster without knowing the actual path..
For Example...
I have 5 nodes all have same actorsytem and formed a cluster.
Each node will have 2 actors in it all actors across cluster have unique name.
I have an actor system outside the cluster.which not part of this cluster.
I have to send message from this actor system to cluster actors.
How can i map respective message to respective actor in cluster each time.Without knowing the actual path of the actor.
Without cluster sharding how actors inside cluster will accessed by outside cluster.
There is a direct answer to your question (at the end), but I think you need to go back to the docs and rethink your design. One of the primary reasons for this is that while what you are asking is possible, it goes against all of the best practices and many of the features involved are deprecated or not recommended. The best practice is that clusters should be self-contained and clusters should expose via well defined APIs and not actor internals. To quote from the remoting docs:
When building an Akka application, you would usually not use the Remoting concepts directly, but instead use the more high-level Akka Cluster utilities or technology-agnostic protocols such as HTTP, gRPC etc.
Essentially the docs are tell you that what you are trying to do is a bad idea. If two actors need to be able to discover each other, they should be in the same cluster. If for some reason you can't have them in the same cluster, for maintainability reasons you should expose the actor in the cluster via REST/gRPC or some other well defined API rather than trying to allow direct access to the actors.
Similarly, in the section about cluster client the docs say:
Cluster Client is deprecated in favor of using Akka gRPC. It is not advised to build new applications with Cluster Client, and existing users should migrate.
But, here's an attempt to answer the question directly. To quote the first sentence of the Actor discovery docs, "There are two general ways to obtain Actor references: by creating actors and by discovery using the Receptionist."
Unrelated side note: (That is a little misleading though, because there are some other ways in untyped Actors, and you also can obviously just be given a ActorRef. It's not the the docs are wrong, I'm just taking them a little out of context.).
Anyway, that leads to the direct answer to your question: ClientClusterReceptionist. You can find details on how to use it in the Classic Cluster Client docs. It's going to require some untyped actors, but since you are using deprecated approaches that probably isn't an issue.
But, moreso, you really should rethink this because based on your last question, it seems like you really aren't understanding how either clustering or sharding work.
As #david-ogren said, what you need is cluster client. You can read the documentation here. With cluster client, you still need to know at least one of the cluster node address and either:
The name of the actor you're trying to communicate with, if you're trying to communicate with the actor instance directly, or
A predefined topic to publish to. If you went for this route, you will need to make sure that all of the participating actors inside the cluster subscribes to the topic to receive them.
You can see a working example in this GitHub repository.
Lets say I have web applicatons/services:
API
Set of Applications
API is used for managing some resources (simple CRUD operations). Now what I need is to subscribe Applications for changes of different API resources. Applications would do some background work on a change.
I came up to idea of callbacks. So that Applications can oauthorise and post to the API a callback config.
I think that this config should look like this:
{
'callback_url': 'http://3rdpartyservice.com/callback',
'resources': ['foo1', 'foo2'],
'ref_data': { 'token': 'abcd1234' }
}
resources is array of the resources that 3rd party service is interested in
ref_data is custom json for 3rd party usage (e.g. for auth)
This way on specified resource change the API would send a request to callback_url. This request would contain resource data, action(create/update/delete) and ref_data.
The intention here is to make this generic enough to allow 3rd party clients configure such callbacks.
So the question are:
Are there any best practices?
What about security potential issues?
Are there any real world examples on the web?
Tx
Sounds very similar as WebHooks or Service Hooks.
Check out the Web Hooks on GitHub, to get a good idea what they are and how they work. See also last alinea Service Hooks, as it explains how github handles these WebHooks. This would be similar for your application. The OAuth explains why and how it is done.
See also Webhooks, REST and the Open Web, from API User Experience.
There is even RestHooks.
The general solution to this requirement is usually called "publish/subscribe". There are dozens of solutions to this - google "publish subscribe REST" for some examples. You can also read "Enterprise Integration Patterns".
They key challenge in this kind of solution is "real-time versus queue".
For instance, if you have an API with a million clients, who are all interested in the same event, you cannot guarantee that in real time you can reach all of those clients within whatever timeframe their application demands. You also have to worry about the network going away, or clients being temporarily down. In this case, you application might define an event queue, and clients look in that queue for events they're interested in. Once you go down that route, you're probably going to use some off-the-shelf software rather than building your own. Apache Camel is a good open source implementation.
In your example, for instance, what happens if you cannot reach 3rdpartyservice.com? Or if http://3rdpartyservice.com/callback throws an error when posting an update to foo1, but not to foo2? Or if http://3rdpartyservice.com/ uses a different flavour of OAuth than you're used to? How do you guarantee http://3rdpartyservice.com/ that it's you who is posting an update, not a hacker?
Your choices really tend to come down to your non-functional requirements, rather than the functional ones - things like uptime, guarantee of notification, guarantee of delivery, etc. are more important than the specifics of how you pass across the parameters, and whether it's "resource-based" or some other protocol.
I've been assigned a small project and directed to use Mirth Connect as part of the solution. We currently do not use Mirth but because we have an upcoming project that will require an interface engine, I was asked to use it for this project so I can gain experience with it. However, I think it's a poor suggestion for this project; I also know my boss would not want me to implement something that adds unnecessary complexity just for the sake of learning.
With that said, I want to make sure I have valid reasons for suggesting that Mirth Connect should not be used for this project. Neither of us know much about it, but I think he's been convinced it is the end all solution for all things interface/webservice related. I appreciate any input I can get from those of you who have more experience with the product than I have.
This is a very simple project in that we have a client needing to make a handful of requests into our system from there's in order to retrieve and update data. For example, they will make a request to get patient demographics, to add an admission for a patient, a request to get a list of possible care settings from our application, etc. For this project we will not use HL7 but a set of predefined XML messages.
Both the client's application and our application reside on the client's network.
They do not want to build any services of their own, so the services we build need to handle all of the work. The results returned in response to their calls to the services will be returned as XML.
There are no plans to integrate any other applications with theirs or ours in the foreseeable future.
It seems to me the best option would be for us to build a standalone web service that would take their request and send back an XML response. I just don't see any reason to include Mirth Connect in the picture (other than for learning but that can be gained in other ways).
What are your thoughts? Is it true that the interface engine is not a good choice if the client wants to receive data from our system without having a receiving mechanism on their end? In other words, they want to make a web service call such as GetCareSettings and to get a response back with an XML representation of all the possible care settings in our system. It seems to me they would need a web service on their end for Mirth to use as a destination to send the results. All Mirth is going to send back is an ACK message, correct? (Unless of course it wrote the data to another webservice on the client end, which they have said they do not want to do.)
Thanks for taking the time to read this. I hope my lack of knowledge and understanding of Mirth Connect and the use of interface engines hasn't made this question difficult to answer.
From what I understand, Your client appears to be either a Lab or a third party service vendor, who will take inputs from your application like patient demographic charts, appointments, provider details etc. Basically he wants to query your application.
A) HL7: It has the capacity to handle query request and response with demographics. I am assuming that you have done you might be knowing about QRY messages.
B) XML/webservices/SOAP:still provides a viable solution, a little more concrete and can be expanded to Handle custom request like GetCallSettings, or may be any other. The vendor is not just interested in fetching patient related data but also other inputs for which HL7 might not be enough.
If we talk about approach, then its a professional advice to use an interface engine. It is not limited to just using mirth connect, you can also use Iguana if you want. A good reason which comes instantly to my mind is that an engine gives you an advantage while troubleshooting, support and maintenance activity.
Your Webservice responses can be handled easily by HTTP sender connector type and through RESTful webservices.
The engine is also capable of handling large volumes of request and responses at the same time, which in case is not required right now, but I think will be the condition later on. Your source in the channel shall change to an Webservice Listener.
Another good approach is to do away with XML and use JSON for handling request and responses, a much more light weighted than XML, to save your overhead with the network. We are doing some similar work, but we are sending request to a webservice through JSON.
Overall, Mirth is there to make your life more easier.
Good Luck!
I am trying to get events when the internet connection is reestablished after it is lost. It is for a data transfer software that I am developing. If I lose the network during data transfer, I would like to be notified when it is back and continue the transfer automatically.
I can of course create a separate thread and check the network once in a while with a timer, but maybe there is a better option out there.
I am developing for windows mainly, in C++ (not .net).
I can also use wxwidgets (I use it for GUI) but I doubt it offers any related functionality.
You might want to check out the System Event Notification Server (SENS) API http://msdn.microsoft.com/en-us/library/cc185680(VS.85).aspx
I have not actually used it, but it seems like it supplies the events your looking for.
EDIT:
WMI appears to have all the information you need about various network connectivity and state changes. It also has an asynchronous event model that can be used to get notifications. The trick is, i suppose, generating the proper WMI query to get the information you want. This blog looks like the right type of query, and this MSDN explains how to handle the events asynchronously.
I don't know which protocol you use and whether you can control the destination, but in that case, the destination can poll for a retry. The destination knows best what it has received, so it can give the received number of bytes as offset for the retransmission.
This MSDN link gives a very detailed example of how to capture events on WMI with COM. The example doesn't actually capture network events - but I believe that if you plug the right query in, it would work.
(lots of code here, so I'm not copying it into the answer)
http://msdn.microsoft.com/en-us/library/aa390425%28v=vs.85%29.aspx
this Codeproject link gives detail on
How to use the Windows NLM API to get notified of new network
connectivity
And maybe helpful to any challenge related to this one.
An application often needs to know if the machine has internet
connectivity and take actions depending on that. In this sample, we
are looking at the usage of the Windows NLM API in managed code so
that an application can choose to respond to internet connectivity
changes. There are many other specific NLM APIs for checking domain
connectivity, network adapter interfaces etc., that haven't been
mentioned in this article; you can refer to this link for further
details. The downloadable zip file has the source code.
more reading here
https://www.codeproject.com/Articles/34650/How-to-use-the-Windows-NLM-API-to-get-notified-of