Is there a Web Service Standard (WS*) for Optimistic Locking / Optimistic Concurrency Control (OCC) designed for interoperability?
There are a number of standards related to pessimistic concurrency control mechanisms, such as WS-AtomicTransaction, but as far as I can tell it doesn't provide functionality for version or timestamp handling related to optimistic locking.
I have found one article on the subject Optimistic locking and WCF, but no mayor standard. Are there such or do you know of other samples or patterns that you would like to recommend?
Related
How do you integrate applications via web services and deal with technical errors like connectivity errors for web service calls which change state?
E.g. when the network connection gets interrupted during a web service call, how does the client know whether the web services has processed its action or not?
Can this issue be solved at the business layer only (e.g. to query a previous call state) or are you aware of some nice frameworks/best practices which can help wrapping transactional guarantees around a web service?
Implementing it all by yourself with some kind of transactional context tracked in the business layer is always an option. You can use some compensation mechanisms to ensure transactions are rolled back if needed, but you'll need to:
have the information on transactions persisted somewhere
use transaction correlation IDs, so you can query when the response has
been lost (having correlation IDs is good idea anyway)
implement the operations needed to read/write/rollback, etc, so it might make your services a bit more complex
Another option I can think of is If you're using SOAP you can go for asynchronous communication and look for some stack implementing WS-Coordination, WS-AtomicTransaction and WS-BusinessActivity specifications, then decide for yourself if it is a good idea in your context or not. For example, I think Axis2 supports these, but of course eventually it depends on technologies and stack you use.
From the article above:
WS-AtomicTransaction defines a coordination type that is most useful
for handling system-generated exceptions, such as an incomplete write
operation or a process terminating abnormally.
Below are the types of 2-Phase Commit that it implements.
Hope this helps!
I have heard enough about RESTful service should be stateless. All state info should be stored in client. And each request should contain all the necessary state info.
But why? What's the benefit of doing that? Only when I know its benefit/motivation can I use it properly.
What if my client have a huge amount of state? Suppose there's an online document editing application. Does client have to send the full text he/she is editing when calling server's RESTful API? Or is this scenario simply not suitable for RESTful approach?
When talking about REST (or well RESTful since not many people adhere 100% to the paper I will quote here) services I always think it's best to start with the source, meaning Fielding dissertation which mentions in 5.1.3 Stateless:
This constraint induces the properties of visibility, reliability, and scalability. Visibility
is improved because a monitoring system does not have to look beyond a single request
datum in order to determine the full nature of the request. Reliability is improved because
it eases the task of recovering from partial failures [133]. Scalability is improved because
not having to store state between requests allows the server component to quickly free
resources, and further simplifies implementation because the server doesn’t have to
manage resource usage across requests.
It goes even further talking about its trade-offs:
Like most architectural choices, the stateless constraint reflects a design trade-off. The
disadvantage is that it may decrease network performance by increasing the repetitive data
(per-interaction overhead) sent in a series of requests, since that data cannot be left on the
server in a shared context. In addition, placing the application state on the client-side
reduces the server’s control over consistent application behavior, since the application
becomes dependent on the correct implementation of semantics across multiple client
versions.
But Fielding doesn't stop even there, he talks about caching to overcome some of the problems.
I highly recommend you go through that PDF, since (from what I remember) that was the original paper that introduced REST.
The use case you provided is a tough one and as many said it depends on your exact scenario. RESTful services are called restFUL and not REST because people found the original paper too limiting and decided to loosen up a bit the rules (for instance the original paper doesn't say anything about batch operations).
The primary benefit is scalability -- by not needing to fetch additional context for each request, you minimize the amount of work done by the server, which may need to service many requests at the same time.
Additionally, it helps provide greater clarity to consumers of your API. By having the user send everything related to the operation being done, they can more clearly see what is actually being done, and the error messages they get can often be more direct as a result; an error can say what value is wrong and why, rather than trying to communicate that something the consumer can't see went wrong on the server.
From the same chapter of Fielding's dissertation:
Like most architectural choices, the stateless constraint reflects a
design trade-off. The disadvantage is that it may decrease network
performance by increasing the repetitive data (per-interaction
overhead) sent in a series of requests, since that data cannot be left
on the server in a shared context.
Advantages are explained as follows:
This constraint induces the properties of visibility, reliability, and
scalability.
Visibility is improved because a monitoring system does
not have to look beyond a single request datum in order to determine
the full nature of the request.
Reliability is improved because it
eases the task of recovering from partial failures [133].
Scalability is improved because not having to store state between requests allows
the server component to quickly free resources, and further simplifies
implementation because the server doesn't have to manage resource
usage across requests.
Regarding your specific case, yes and no. This is how the Web works. When we edit something online, we send entire request to the server. Though it is a design choice how we implement partial updates.
Software can be designed to accomplish this goal by sending PUT/POST requests to sub-resources. For example:
PUT /book/chapter1 HTTP/1.1
PUT /book/chapter2 HTTP/1.1
PUT /book/chapter3 HTTP/1.1
instead of updating whole resource:
PUT /book HTTP/1.1
Content-Type: text/xyz
Content-Length: ...
We would like to run our cms::MessageConsumer and cms::MessageProducer on different threads of the same process.
How do we do this safely?
Would having two cms::Connection objects and two cms::Session objects, one each for consumer and producer, be sufficient to guarantee safety? Is this necessary?
Is there shared state between objects at the static library level that would prevent this type of usage?
You should read the JMS v1.1 specification, it calls out clearly which objects are valid to use in multiple threads and which are not. Namely the Session, MessageConsumer and MessageProducer are considered unsafe to share amongst threads. We generally try to make them as thread safe as we can but there are certainly ways in which you can get yourself into trouble. Its generally a good idea to use a single session in each thread and in general its a good idea to use a session for each MessageConsumer / MessageProducer since the Session contains a single dispatch thread which means that a session with many consumers must share its dispatch thread for sending messages on to each consumer which can lower latency depending on the scenario.
I'm answering my own question to supplement Tim Bish's answer, which I am accepting as having provided the essential pieces of information.
From http://activemq.apache.org/cms/cms-api-overview.html
What is CMS?
The CMS API is a C++ corollary to the JMS API in Java which is used to
send and receive messages from clients spread out across a network or
located on the same machine. In CMS we've made every attempt to
maintain as much parity with the JMS api as possible, diverging only
when a JMS feature depended strongly on features in the Java
programming language itself. Even though there are some differences
most are quite minor and for the most part CMS adheres to the JMS
spec, so having a firm grasp on how JMS works should make using CMS
that much easier.
What does the JMS spec say about thread safety?
Download spec here:
http://download.oracle.com/otndocs/jcp/7195-jms-1.1-fr-spec-oth-JSpec/
2.8 Multithreading JMS could have required that all its objects support concurrent use. Since support for concurrent access typically
adds some overhead and complexity, the JMS design restricts its
requirement for concurrent access to those objects that would
naturally be shared by a multithreaded client. The remainder are
designed to be accessed by one logical thread of control at a time.
JMS defines some specific rules that restrict the concurrent use of
Sessions. Since they require more knowledge of JMS specifics than we
have presented at
Table 2-2 JMS Objects that Support Concurrent Use
Destination: YES
ConnectionFactory: YES
Connection: YES
Session: NO
MessageProducer: NO
MessageConsumer: NO
this point, they will be described later. Here we will describe the
rationale for imposing them.
There are two reasons for restricting concurrent access to Sessions.
First, Sessions are the JMS entity that supports transactions. It is
very difficult to implement transactions that are multithreaded.
Second, Sessions support asynchronous message consumption. It is
important that JMS not require that client code used for asynchronous
message consumption be capable of handling multiple, concurrent
messages. In addition, if a Session has been set up with multiple,
asynchronous consumers, it is important that the client is not forced
to handle the case where these separate consumers are concurrently
executing. These restrictions make JMS easier to use for typical
clients. More sophisticated clients can get the concurrency they
desire by using multiple sessions.
As far as I know from the Java side, the connection is thread safe (and rather expensive to create) but Session and messageProducer are not thread safe. Therefore it seems you should create a Session for each of your threads.
Every description on the Message Bus architecture seems to describe a centralised intermediary for routing messages:
Each application has a single connection to the message bus instead
of multiple dedicated connections to each of the other applications.
http://msdn.microsoft.com/en-us/library/ff647328.aspx
Structure the connecting middleware between these applications as a
Message Bus that enables them to work together using messaging.
http://www.eaipatterns.com/MessageBus.html
Service Bus is messaging infrastructure that sits between applications
allowing them to exchange messages in a loosely coupled way for
improved scale and resiliency.
http://www.windowsazure.com/en-us/home/features/messaging/
The descriptions seem identical to the Message Broker architecture; leading me to believe the following:
The term 'Message Bus' is effectively the same as 'MoM' - a generic term for referring to a messaging infrastructure without explicitly mentioning a topology. I.e. Message Broker is a type of Message Bus.
This seems like a controversial topic, and whilst there's a few similar questions on SO, none have (IMO) definitive answers.
P.s. I am aware that Enterprise Service Bus is entirely different, and is effectively a 'feature rich' MoM.
You are right, this area of terminology is very messy and there are no definitive answer, but this is my view from my professional life:
The first thing is to do is to think about this in different levels of abstractions - architectural concepts vs products.
A message bus (http://www.eaipatterns.com/MessageBus.html) is often referd to a as an architectural princip to decouple applications often together with canonical data formats (http://www.eaipatterns.com/CanonicalDataModel.html), much regardless of technology. Often implemented using MoM products.
A MoM typically refers to a family of middleware products that implements asynchronous message passing. Often built supporting standards such as JMS and AMQP (typically WebSphere MQ, ActiveMQ, Azure messaging, hornetQ, RabbitMQ, etc.etc.).
A message broker is often a MoM, and both terms are used for the same messaging product. There might be confusion here, since the main ESB product of IBM is called WebSphere Message Broker while it rather an ESB used side by side with the MoM product - WebSphere MQ. Usually most MoM call themselves brokers, such as RabbitMQ, ActiveMQ, QPid etc.
Hope this did not cause more confusion :)
The differences are mainly different vendors trying to differentiate themselves in the Market and increase their mind-share. Since vendors typically defined their own terms, there will usually be a slight differences in what each vendor means by the terms they use.
However in general they are the same class or type of system. They organise and manage the transport, queueing and persistence (storage) of messages from one software service to another, beyond that, you need to be looking at vendors specific feature lists for their product.
WebSphere Message Broker belongs to the ESB layer of an SOA and it is also MOM. WebSphere MQ is also MOM. I think a lot of the confusion is around abstract architecture concepts and actual software products. SOA and ESB are abstract concepts. The whole enterprise architecture industry IBM, Microsoft, Oracle, TIBCO etc. are always vying for market share. Software companies often pretend they invented something new and better in order to get attention from their customers. They make a lot of wild promises about their products when in reality they create a whole new set of expensive problems (e.g. XML transformations cause serious performance and security issues). Because it is corporate greed driven and "get to market fast" there is always an abundance of confusion and IT problems and much stress for IT workers.
We are using messaging in a project of ours to implement pessimistic concurrency. This means that if messaging goes down (channel goes down), concurrency goes down.
Is this done in other business applications?
Do you close the application (log out the user) if messaging goes down?
I'm thinking more of combining the optimistic and pessimistic concurrency. Then if pessimistic concurrency goes down, there's still a backup optimistic concurrency...
thx, Lieven Cardoen
As usual, I think the answer depends on the nature of the business application that you are building. What are the SLAs for your application? How mission critical is it?
If your messaging infrastructure faults, does the application continue to function aside from the lock service? If so, then you probably have an obligation to make sure your concurrency control mechanism isn't the single point of failure.
Furthermore, the topic of achieving a truly distributed, fault-tolerant pessimistic locking mechanism requires one to address the problem of consensus. Most pessimistic locking algorithms rely on there being a single, serialized authority that can respond to requests for locks (i.e. there's a "lock" table or perhaps there's a singleton lock server).
Such a design has single point of failure written all over it. To answer your first question--Yes, I've seen business applications use messaging to provide pessimistic locking. However, fully solving the fault tolerance problem seems like overkill for most business applications I've encountered.
Optimistic concurrency control does not have this problem by its nature which is why it's generally preferred in distributed, fault-tolerant applications. However, I realize that business requirements frequently win out over ease-of-implementation.
If the topic interests you, Google has published an article on their Chubby Lock Service which leverages the Paxos consensus protocol.