I am looking for some solid arguements against a solution supplied where a public facing webserver hosts an aspx form and based on user input places the content of the form in XML in an email body and sends it to an email address only used for this solution. Then an internal system behind the company firewall reads the XML after retrieving the email from the email server and processes from there. I dont think this will be a robust solution and concerned about maintaining it so would just rather replace it now but there is pressure to keep solution.
Thanks
You mostly can't judge an architectural solution without knowing the specific constraints.
Under certain constraints, this may be very well be the best solution.
Let's take a look at the weak points first:
Messages may be lost due to the mail service not available.
Messages may be too big for the mail service. (In my corp we have a limit of 10Mb for instance.)
Messages may be corrupted on transfer. (Mail service may apply virus scanners and boast with this fact, add footers, rename attachments etc.)
Mail system may not cope with the additional burden, if the message traffic is too big.
Order of delivery is not guaranteed.
The solution is somewhat non-conventional.
Security and other non-func may be not fulfilled.
On the other hand:
This is probably a (very pragmatic) implementation of asynchronous messaging. Asynchronous messaging is often much more powerful and reliably than synchronous solutions.
This solution uses an already existing infrastructure.
Mail system does normally not lose messages "just so". So we basically have a reliable persistent message storage here.
Mail systems are often considered to be "mission critical" so they're often built highly reliable and redundant. So using the mail service may be actually more reliable than introducing a new software/hardware component.
And cheaper.
Can be tested with very pragmatic means.
E-mail has good library support.
You don't need an expensive professional for implementation.
So imagine the following constraints:
Build an asynchronous message processing.
Losing small percentage of messages is not a big deal.
Do it fast.
Do it cheap.
Quick and dirty is OK ("we'll throw it away in three month anyway").
Under these constraints this might be a very good and pragmatic solution.
To address the point by #techtrek:
"far more robust" - see above, mail system may actually be more reliable than an internal ESB infrastructure. At least this is my experience.
Agree, but not THAT risky. Attachments are normally NOT damaged in anyway. Otherwise management would scream every time their PowerPoint slides get corrupted.
Emails service down - well, ESB or any internal service may go down as well.
I don't quite understand why e-mail traceability is more complicated. I send an email, it either arrives or not. If not then this is a problem of the mail service. "Complicating" compared to what?
Well, of course mail service administration is separate, why is this a maintenance headache? We actually have all the platform services (databases, servers, ESB, etc.) administered and maintained by separate teams. This is a normal practice, I don't see why it should be a problem here. On the contrary, with the mail service you probably have a professional team specifically dedicated to the reliability of that transport channel.
Frankly, I saw quite a few ESB/MQ-solutions where I really thought that it would be MUCH cheaper, easier and in the fact more reliable if a few distinct apps would just send each other e-mails.
The issue with using email as the relay agent is that:
Creating a webservice that allows your company's internal system to directly intercept and parse XML system seems far more robust to me.
Encapsulating the XML mime type within a transmission protocol (Email) is risky in and of itself.
As a result of (2), there are two points of failure (corruption in the xml conversion process) and also the risk of the email service going down.
Beyond just points of failure, you're also complicating traceability by an order of magnitude.
Email administration is often separate to the administration of the web-service. Unless there are real, legitimate reaosns for this, it just sounds like more maintenance headache?
I agree with what has been said, especially on point 4. Application and eMail Maintenance might be disconnected entities.
Another aspect to consider is the possibility ti send mail from anywhere to the backend and flood it this way
Related
I am doing some research on SOAP, for a personal project, and I came across a website with a list of pros and cons for using SOAP, and I understood what most of them meant, except for this one under disadvantages:
SOAP is typically limited to pooling, and not event notifications, when leveraging HTTP for transport. What's more, only one client can use the services of one server in typical situations.
From my understanding of pooling, there should be no issue pooling a SOAP Object for re usability. Pooling is simply a way to use the same resources over and over again, like a connection to a database. Also not entirely certain on the context of Event Notifications.
So my two questions here are, what does the above block quoted text actually mean, and is this information correct?
Website: http://searchsoa.techtarget.com/definition/SOAP
SOAP is RPC, and in RPC some local client invokes a method on some remote target and receives a result. That's how it works, so SOAP works that way too. A client invokes a service asking for something and the service just responds.
If you want "events" in this type of communication the most simple approach is to invoke the service more often (i.e. polling). This has the advantage that nothing changes for the server or the client. It's the same RPC call but done more frequently.
These days everyone is connected to the web and everyone is subscribed to all sorts of services. They want to get notified as soon as something happens to the world around them. Pooling becomes inefficient in this sea of users and services because you are wasting resources. You might poll a service a hundred times just to get back one notification. For this reason technology is evolving so that resource use is minimized. And the direction this is moving to is push services.
Now almost everything happens in the browser. Every browser manufacturer rushes to implement the latest technology changes and HTML5 spec. This means actual pages that push notifications to users instead of faking it with Ajax, comet, etc.
SOAP has been around since 1998 and it's not moving as fast as the rest of the web, mainly because SOAP is mostly an enterprise player and because it's a protocol. Because it's a protocol you have to make new technology available to it without breaking that protocol. Things move slower so people have abandoned SOAP in favor of other ways of doing server-client communication.
SOAP is typically limited to pooling, and not event notifications...
That is correct. But be aware that "typically" does not mean "always".
You can have events, but it's harder. It involves using WS-* specifications like WS-Eventing and WS-Addressing. This is a change in the way SOAP clients operate because a client now becomes some sort of a service too because it needs to receive calls too, not just initiate them. If your technology stack implements these specifications then good for you, but if it doesn't, then you have to build it yourself and it's a real pain.
So for these reasons, if you don't have blocking performance or resource usage issues, you "typically" chose doing polling with SOAP and not event notifications.
I have heard enough about RESTful service should be stateless. All state info should be stored in client. And each request should contain all the necessary state info.
But why? What's the benefit of doing that? Only when I know its benefit/motivation can I use it properly.
What if my client have a huge amount of state? Suppose there's an online document editing application. Does client have to send the full text he/she is editing when calling server's RESTful API? Or is this scenario simply not suitable for RESTful approach?
When talking about REST (or well RESTful since not many people adhere 100% to the paper I will quote here) services I always think it's best to start with the source, meaning Fielding dissertation which mentions in 5.1.3 Stateless:
This constraint induces the properties of visibility, reliability, and scalability. Visibility
is improved because a monitoring system does not have to look beyond a single request
datum in order to determine the full nature of the request. Reliability is improved because
it eases the task of recovering from partial failures [133]. Scalability is improved because
not having to store state between requests allows the server component to quickly free
resources, and further simplifies implementation because the server doesn’t have to
manage resource usage across requests.
It goes even further talking about its trade-offs:
Like most architectural choices, the stateless constraint reflects a design trade-off. The
disadvantage is that it may decrease network performance by increasing the repetitive data
(per-interaction overhead) sent in a series of requests, since that data cannot be left on the
server in a shared context. In addition, placing the application state on the client-side
reduces the server’s control over consistent application behavior, since the application
becomes dependent on the correct implementation of semantics across multiple client
versions.
But Fielding doesn't stop even there, he talks about caching to overcome some of the problems.
I highly recommend you go through that PDF, since (from what I remember) that was the original paper that introduced REST.
The use case you provided is a tough one and as many said it depends on your exact scenario. RESTful services are called restFUL and not REST because people found the original paper too limiting and decided to loosen up a bit the rules (for instance the original paper doesn't say anything about batch operations).
The primary benefit is scalability -- by not needing to fetch additional context for each request, you minimize the amount of work done by the server, which may need to service many requests at the same time.
Additionally, it helps provide greater clarity to consumers of your API. By having the user send everything related to the operation being done, they can more clearly see what is actually being done, and the error messages they get can often be more direct as a result; an error can say what value is wrong and why, rather than trying to communicate that something the consumer can't see went wrong on the server.
From the same chapter of Fielding's dissertation:
Like most architectural choices, the stateless constraint reflects a
design trade-off. The disadvantage is that it may decrease network
performance by increasing the repetitive data (per-interaction
overhead) sent in a series of requests, since that data cannot be left
on the server in a shared context.
Advantages are explained as follows:
This constraint induces the properties of visibility, reliability, and
scalability.
Visibility is improved because a monitoring system does
not have to look beyond a single request datum in order to determine
the full nature of the request.
Reliability is improved because it
eases the task of recovering from partial failures [133].
Scalability is improved because not having to store state between requests allows
the server component to quickly free resources, and further simplifies
implementation because the server doesn't have to manage resource
usage across requests.
Regarding your specific case, yes and no. This is how the Web works. When we edit something online, we send entire request to the server. Though it is a design choice how we implement partial updates.
Software can be designed to accomplish this goal by sending PUT/POST requests to sub-resources. For example:
PUT /book/chapter1 HTTP/1.1
PUT /book/chapter2 HTTP/1.1
PUT /book/chapter3 HTTP/1.1
instead of updating whole resource:
PUT /book HTTP/1.1
Content-Type: text/xyz
Content-Length: ...
I've been assigned a small project and directed to use Mirth Connect as part of the solution. We currently do not use Mirth but because we have an upcoming project that will require an interface engine, I was asked to use it for this project so I can gain experience with it. However, I think it's a poor suggestion for this project; I also know my boss would not want me to implement something that adds unnecessary complexity just for the sake of learning.
With that said, I want to make sure I have valid reasons for suggesting that Mirth Connect should not be used for this project. Neither of us know much about it, but I think he's been convinced it is the end all solution for all things interface/webservice related. I appreciate any input I can get from those of you who have more experience with the product than I have.
This is a very simple project in that we have a client needing to make a handful of requests into our system from there's in order to retrieve and update data. For example, they will make a request to get patient demographics, to add an admission for a patient, a request to get a list of possible care settings from our application, etc. For this project we will not use HL7 but a set of predefined XML messages.
Both the client's application and our application reside on the client's network.
They do not want to build any services of their own, so the services we build need to handle all of the work. The results returned in response to their calls to the services will be returned as XML.
There are no plans to integrate any other applications with theirs or ours in the foreseeable future.
It seems to me the best option would be for us to build a standalone web service that would take their request and send back an XML response. I just don't see any reason to include Mirth Connect in the picture (other than for learning but that can be gained in other ways).
What are your thoughts? Is it true that the interface engine is not a good choice if the client wants to receive data from our system without having a receiving mechanism on their end? In other words, they want to make a web service call such as GetCareSettings and to get a response back with an XML representation of all the possible care settings in our system. It seems to me they would need a web service on their end for Mirth to use as a destination to send the results. All Mirth is going to send back is an ACK message, correct? (Unless of course it wrote the data to another webservice on the client end, which they have said they do not want to do.)
Thanks for taking the time to read this. I hope my lack of knowledge and understanding of Mirth Connect and the use of interface engines hasn't made this question difficult to answer.
From what I understand, Your client appears to be either a Lab or a third party service vendor, who will take inputs from your application like patient demographic charts, appointments, provider details etc. Basically he wants to query your application.
A) HL7: It has the capacity to handle query request and response with demographics. I am assuming that you have done you might be knowing about QRY messages.
B) XML/webservices/SOAP:still provides a viable solution, a little more concrete and can be expanded to Handle custom request like GetCallSettings, or may be any other. The vendor is not just interested in fetching patient related data but also other inputs for which HL7 might not be enough.
If we talk about approach, then its a professional advice to use an interface engine. It is not limited to just using mirth connect, you can also use Iguana if you want. A good reason which comes instantly to my mind is that an engine gives you an advantage while troubleshooting, support and maintenance activity.
Your Webservice responses can be handled easily by HTTP sender connector type and through RESTful webservices.
The engine is also capable of handling large volumes of request and responses at the same time, which in case is not required right now, but I think will be the condition later on. Your source in the channel shall change to an Webservice Listener.
Another good approach is to do away with XML and use JSON for handling request and responses, a much more light weighted than XML, to save your overhead with the network. We are doing some similar work, but we are sending request to a webservice through JSON.
Overall, Mirth is there to make your life more easier.
Good Luck!
I've been tasked with creating an intermediate layer which needs to exchange data (over HTTP) between two independent systems (e.g. Receiver <=> Intermediate Layer (IL) <=> Sender). Receiver and Sender both expose a set of API's via Web Services. Everytime a transaction occurs in the Sender system, the IL should know about it (I'm thinking of creating a Windows Service which constantly pings the Sender), massage the data, then deliver it to the Receiver. The IL can temporarily store the data in a SQL database until it is transferred to the Receiver. I have the following questions -
Can WCF (haven't used it a lot) be used to talk to the Sender and Receiver (both expose web services)?
How do I ensure guaranteed delivery?
How do I ensure security of the messages over the Internet?
What are best practices for handling concurrency issues?
What are best practices for error handling?
How do I ensure reliability of the data (data is not tampered along the way)
How do I ensure the receipt of the data back to the Sender?
What are the constraints that I need to be aware of?
I need to implement this on MS platform using a custom .NET solution. I was told not to use any middleware like BizTalk. The receiver is an SDFC instance, if that matters.
Any pointers are greatly appreciated. Thank you.
A Windows Service that orchestras the exchange sounds fine.
Yes WCF can deal with traditional Web Services.
How do I ensure guaranteed delivery?
To ensure delivery you can use TransactionScope to handle the passing of data between the
Receiver <=> Intermediate Layer and Intermediate Layer <=> Sender but I wouldn't try and do them together.
You might want to consider some sort of queuing mechanism to send the data to the receiver; I guess I'm thinking more of a logical queue rather than an actual queuing component. A workflow framework could also be an option.
make sure you have good logging / auditing in place; make sure it's rock solid, has the right information and is easy to read. Assuming you write a service it will execute without supervision so the operational / support aspects are more demanding.
Think about scenarios:
How do you manage failed deliveries?
What happens if the reciever (or sender) is unavailbale for periods of time (and how long is that?); for example: do you need to "escalate" to an operator via email?
How do I ensure security of the messages over the Internet?
HTTPS. Assuming other existing clients make calls to the Web Services how do they ensure security? (I'm thinking encryption).
What are best practices for handling concurrency issues?
Hmm probably a separate question. You should be able to find information on that easily enough. How much data are we taking? what sort of frequency? How many instances of the Windows Service were you thinking of having - if one is enough why would concurrency be an issue?
What are best practices for error handling?
Same as for concurrency, but I can offer some pointers:
Use an established logging framework, I quite like MS EntLibs but there are others (re-using whatever's currently used is probably going to make more sense - if there is anything).
Remember that execution is unattended so ensure information is complete, clear and unambiguous. I'd be tempted to log more and dial it down once a level of comfort is reached.
use a top level handler to ensure nothing get's lost; but don;t be afraid to log deep in the application where you can still get useful context (like the metadata of the data being sent / recieved).
How do I ensure the receipt of the data back to the Sender?
Include it (sending the receipt) as a step that is part of the transaction.
On a different angle - have a look on CodePlex for ESB type libraries, you might find something useful: http://www.codeplex.com/site/search?query=ESB&ac=8
For example ESBasic which seems to be a class library which you could reuse.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I'm not a security expert by any means, but I favor creating REST-style web services.
In creating a new service which needs to have the data it transmits secure. We've entered a debate over which approach is more secure - REST with HTTPS or a SOAP WS with WS-Security.
I am under the impression we could use HTTPS for all the web service calls and this approach would be secure. The way I look at it is, "if HTTPS is good enough for bank and financial web sites, it's good enough for me". Again, I'm not expert in this space, but I'd think that these people have thought considerably hard about this problem and are comfortable with HTTPS.
A coworker disagrees and says SOAP and WS-Security is the only way to go.
The web seems all over the board on this.
Maybe the community here could weigh in on the pros and cons of each? Thanks!
HTTPS secures the transmission of the message over the network and provides some assurance to the client about the identity of the server. This is what's important to your bank or online stock broker. Their interest in authenticating the client is not in the identity of the computer, but in your identity. So card numbers, user names, passwords etc. are used to authenticate you. Some precautions are then usually taken to ensure that submissions haven't been tampered with, but on the whole whatever happens over in the session is regarded as having been initiated by you.
WS-Security offers confidentiality and integrity protection from the creation of the message to it's consumption. So instead of ensuring that the content of the communications can only be read by the right server it ensures that it can only be read by the right process on the server. Instead of assuming that all the communications in the securely initiated session are from the authenticated user each one has to be signed.
There's an amusing explanation involving naked motorcyclists here:
https://learn.microsoft.com/archive/blogs/vbertocci/end-to-end-security-or-why-you-shouldnt-drive-your-motorcycle-naked
So WS-Security offers more protection than HTTPS would, and SOAP offers a richer API than REST. My opinion is that unless you really need the additional features or protection you should skip the overhead of SOAP and WS-Security. I know it's a bit of a cop-out but the decisions about how much protection is actually justified (not just what would be cool to build) need to be made by those who know the problem intimately.
REST security is transport dependent while SOAP security is not.
REST inherits security measures from the underlying transport while SOAP defines its own via WS-Security.
When we talk about REST, over HTTP - all security measures applied HTTP are inherited and this is known as transport level security.
Transport level security, secures your message only while its on the wire - as soon as it leaves the wire, the message is no more secured.
But, with WS-Security, its message level security - even though the message leaves the transport channel it will be still protected. Also - with message level security you can partly encrypt the message [not the entire message, but only the parts you want] - but with transport level security you can't do it.
WS-Security has measures for authentication, integrity, confidentiality and non-repudiation while SSL doesn't support non repudiation [with 2-legged OAuth it does].
In performance-wise SSL is very much faster than WS-Security.
Thanks...
Technically, the way you have it worded, neither is correct, because the SOAP method's communication isn't secure, and the REST method didn't say anything about authenticating legitimate users.
HTTPS prevents attackers from eavesdropping on the communication between two systems. It also verifies that the host system (server) is actually the host system the user intends to access.
WS-Security prevents unauthorized applications (users) from accessing the system.
If a RESTful system has a way of authenticating users and a SOAP application with WS-Security is using HTTPS, then really both are secure. It's just a different way of presenting and accessing data.
See the wiki article:
In point-to-point situations confidentiality and data integrity can also be enforced on Web services through the use of Transport Layer Security (TLS), for example, by sending messages over https.
WS-Security however addresses the wider problem of maintaining integrity and confidentiality of messages until after a message was sent from the originating node, providing so called end to end security.
That is:
HTTPS is a transport layer (point-to-point) security mechanism
WS-Security is an application layer (end-to-end) security mechanism.
As you say, REST is good enough for banks so should be good enough for you.
There are two main aspects to security: 1) encryption and 2) identity.
Transmitting in SSL/HTTPS provides encryption over the wire. But you'll also need to make sure that both servers can confirm that they know who they are speaking to. This can be via SSL client certificates, shares secrets, etc.
I'm sure one could make the case that SOAP is "more secure" but probably not in any significant way. The nude motorcyclist analogy is cute but if accurate would imply that the whole internet is insecure.
I don't yet have the rep needed to add a comment or I would have just added this to Bell's answer. I think Bell did a very good job of summing up the top level pros and cons of the two approaches. Just a few other factors that you might want to consider:
1) Do the requests between your clients and your service need to go through intermediaries that require access to the payload? If so then WS-Security might be a better fit.
2) It is actually possible to use SSL to provide the server with assurance as to the clients identity using a feature called mutual authentication. However, this doesn't get much use outside of some very specialized scenarios due to the complexity of configuring it. So Bell is right that WS-Sec is a much better fit here.
3) SSL in general can be a bit of a bear to setup and maintain (even in the simpler configuration) due largely to certificate management issues. Having someone who knows how to do this for your platform will be a big plus.
4) If you might need to do some form of credential mapping or identity federation then WS-Sec might be worth the overhead. Not that you can't do this with REST, you just have less structure to help you.
5) Getting all the WS-Security goop into the right places on the client side of things can be more of a pain than you would think it should.
In the end though it really does depend on a lot of things we're not likely to know. For most situations I would say that either approach will be "secure enough" and so that shouldn't be the main deciding factor.
Brace yourself, here there's another coming :-)
Today I had to explain to my girlfriend the difference between the expressive power of WS-Security as opposed to HTTPS. She's a computer scientist, so even if she doesn't know all the XML mumbo jumbo she understands (maybe better than me) what encryption or signature means. However I wanted a strong image, which could make her really understand what things are useful for, rather than how they are implemented (that came a bit later, she didn't escape it :-)).
So it goes like this. Suppose you are naked, and you have to drive your motorcycle to a certain destination.
In the (A) case you go through a transparent tunnel: your only hope of not being arrested for obscene behaviour is that nobody is looking. That is not exactly the most secure strategy you can come out with... (notice the sweat drop from the guy forehead :-)). That is equivalent to a POST in clear, and when I say "equivalent" I mean it.
In the (B) case, you are in a better situation. The tunnel is opaque, so as long as you travel into it your public record is safe. However, this is still not the best situation. You still have to leave home and reach the tunnel entrance, and once outside the tunnel probably you'll have to get off and walk somewhere... and that goes for HTTPS. True, your message is safe while it crosses the biggest chasm: but once you delivered it on the other side you don't really know how many stages it will have to go through before reaching the real point where the data will be processed. And of course all those stages could use something different than HTTP: a classical MSMQ which buffers requests which can't be served right away, for example. What happens if somebody lurks your data while they are in that preprocessing limbo? Hm. (read this "hm" as the one uttered by Morpheus at the end of the sentence "do you think it's air you are breathing?").
The complete solution (c) in this metaphor is painfully trivial: get some darn clothes on yourself, and especially the helmet while on the motorcycle!!! So you can safely go around without having to rely on opaqueness of the environments. The metaphor is hopefully clear: the clothes come with you regardless of the mean or the surrounding infrastructure, as the messsage level security does. Furthermore, you can decide to cover one part but reveal another (and you can do that on personal basis: airport security can get your jacket and shoes off, while your doctor may have a higher access level), but remember that short sleeves shirts are bad practice even if you are proud of your biceps :-) (better a polo, or a t-shirt).
I'm happy to say that she got the point! I have to say that the clothes metaphor is very powerful: I was tempted to use it for introducing the concept of policy (disco clubs won't let you in sport shoes; you can't go to withdraw money in a bank in your underwear, while this is perfectly acceptable look while balancing yourself on a surf; and so on) but I thought that for one afternoon it was enough ;-)
Architecture - WS, Wild Ideas
Courtesy : http://blogs.msdn.com/b/vbertocci/archive/2005/04/25/end-to-end-security-or-why-you-shouldn-t-drive-your-motorcycle-naked.aspx
I work in this space every day so I want to summarize the good comments on this in an effort to close this out:
SSL (HTTP/s) is only one layer ensuring:
The server being connected to presents a certificate proving its
identity (though this can be spoofed through DNS poisoning).
The communications layer is encrypted (no eavesdropping).
WS-Security and related standards/implementations use PKI that:
Prove the identity of the client.
Prove the message was not modified
in-transit (MITM).
Allows the server to authenticate/authorize the
client.
The last point is important for service requests when the identity of the client (caller) is paramount to knowing IF they should be authorized to receive such data from the service.
Standard SSL is one-way (server) authentication and does nothing to identify the client.
The answer actually depends on your specific requirements.
For instance, do you need to protect your web messages or confidentiality is not required and all you need is to authenticate end parties and ensure message integrity? If this is the case - and it often is with web services - HTTPS is probably the wrong hammer.
However - from my experience - do not overlook the complexity of the system you're building. Not only HTTPS is easier to deploy correctly, but an application that relies on the transport layer security is easier to debug (over plain HTTP).
Good luck.
REST Over HTTPS Should be a secure method as long as API provider implements authorization a server end. In a case of web application as well what we do is accessing a web application via HTTPS and some authentication/authorization, traditionally web applications did not have security issues then Restful API would also counter security issues without problem !
If your RESTFul call sends XML Messages back and forth embedded in the Html Body of the HTTP request, you should be able to have all the benefits of WS-Security such as XML encryption, Cerificates, etc in your XML messages while using whatever security features are available from http such as SSL/TLS encryption.