My understanding of Akka is that it provides a model whereby multiple, isolated threads can communicate with each other in a highly concurrent fashion. It uses the "actor model", where each thread is an "actor" with a specific job to do. You can orchestrate which messages get passed to which actors under what conditions.
I've used Camel before, and to me, I feel like it's sort of lost its luster/utility now that Akka is so mature and well documented. As I understand it, Camel is about enterprise integration, that is, integrating multiple disparate systems together, usually in some kind of service bus fashion.
But think about it: if I am currently using Camel to:
Poll an FTP server for a file, and once found...
Transform the contents of that file into a POJO, then...
Send out an email if the POJO has a certain state, or
Persist the POJO to a database in all other cases
I can do the exact same thing with Akka; I can have 1 Actor for each of those steps (Poll FTP, transform file -> POJO, email or persist), wire them together, and let Akka handle all the asynchrony/concurrency.
So even though Akka is a concurrency framework (using actors), and even though Camel is about integration, I have to ask: Can't Akka solve everything that Camel does? In ther words: What use cases still exist to use Camel over Akka?
Akka and Camel are two different beasts (besides one is a mountain and one is an animal).
You mention it yourself:
Akka is a tool to implement the reactor pattern, i.e. a message based concurrency engine for potentially distributed systems.
Camel is a DSL/framwork to implement Enterprise Integration Patterns.
There are a lot of things that would be pretty though in Akka, that is easy in Camel. Transactions for sure. Then all the logic various transport logic and options, as Akka does not have an abstraction for an integration message. Then there are well developed EIPs that are great in Camel, the multicast, splitting, aggregation, XML/JSON handling, text-file parsing, HL7, to mention a only a few. Of course, you can do it all in pure java/scala, but that's not the point. The point is to be able to describe the integration using a DSL, not to implement the underlying logic once again.
Non the less, Akka TOGETHER with Camel is pretty interesting. Especially using Scala. Then you have EIP on top of the actor semantics, which is pretty powerful in the right arena.
Example from akka.io
import akka.actor.Actor
import akka.camel.{ Producer, Oneway }
import akka.actor.{ ActorSystem, Props }
class Orders extends Actor with Producer with Oneway {
def endpointUri = "jms:queue:Orders"
}
val sys = ActorSystem("some-system")
val orders = sys.actorOf(Props[Orders])
orders ! <order amount="100" currency="PLN" itemId="12345"/>
A full sample/tutorial can be found at typesafe.
Related
I’m having problem to grasp how to do TDD when building a client-server system.
The simple katas (Fizzbuzz etc) are easy to understand, but when my client needs to send the server a file using TCP sockets and get a response back from the server I’m getting confused regarding how to test that.
I had a project building a file-sync system. The client will monitor a folder and every time a change happens (new file, file deletion etc.) the server should update automatically.
The client can have many devices, for example I can have a copy of the folder in two different computers and they all should sync perfectly.
I started the project with tests, but once I reached the part of speaking with the server I got stuck and didn’t understand how to implement tests.
Most of the things I find regarding TDD are the simple stuff. I would love your advice on this slightly more complex application.
I’m having problem to grasp how to do TDD when building a client-server system.
The reference you want is Growing Object Oriented Software, Guided by Tests
I started the project with tests, but once I reached the part of speaking with the server I got stuck and didn’t understand how to implement tests.
Basic idea: you are trying to work towards a design where you can separate the complicated code from the code that is hard/expensive to test.
This often means three "modules"
A really simple module that knows how to talk to the network
A complicated module that knows how to prepare messages for the network, and how to interpret the responses (and timeouts)
A module that can coordinate the interaction of the two modules above.
The first module, you "test" using code review, acceptance testing, and taking advantage of the fact that it doesn't change very often (because it is so simple).
The second module, you use lots of programmer tests to make sure that the logic correctly handles all of the different messages that can pass through it
The third module, you concentrate on testing the protocol. Here, we'll often use a substitute implementation (aka a mock or some other flavor of test double) for one or both of the first two modules.
In a language with types like Java or C#, the need for substitutes will often mean that the first two modules will need to implement some interface, and the third module will depend on those interfaces rather than having direct dependencies on the implementations.
You'll likely also need some code in your composition root that wires together the actual implementations.
For a good take on separating the networking client from the networking logic/protocol, see Cory Benfield 2016.
It might also be useful to review:
Boundaries, by Gary Bernhardt
At the Boundaries, Applications Aren't Object Oriented by Mark Seemann
Put the client code that is working with the socket into a separate class that can be injected in the "business code". For your tests inject a mock instead, verifying that the API of the "client socket adapter" is called in the apropriate way. Mocking libraries make this easy.
Put the server code that is working with the socket into a separate class and design an internal API for the "business code" that the "server socket adapter" is calling. Ignore the adapter in your tests and call the API of the business code directly.
You might want to read about the Ports & Adapter architecture (sometimes also called the "Hexagonal Model").
At the moment I am writing this question, there are not (yet) any questions tagged with both [apache-kafka] and [xslt].
I am a "classic" Message oriented middleware (BizTalk, TIBCO, ...) guy who is just discovering Kafka and its IMPRESSIVE performance figures!
And, then, I am wondering what the recommendation from the "Kafka-community" about how to transform the message payload between its publishing and its consumption...
Indeed, in my world of integration, the data structure (i.e. format) exposed by the producer is usually radically different from the data structure (format) expected by the consumer. As an example, I may have, as a producer, a mainframe application formatting data in a COBOL copybook structure while my front-end application wants to consume a modern JSON format.
[Update following the 1st answer from #morganw09dev]
I like the proposal from #morganw09dev, but I am a bit "annoyed" by the creation of consumer-specific topics. I see the "Topic B" (see #morganw09dev's 1st answer) as the topic specific for my front-end application in order for consuming information from the "Topic A". In order words, this specificity makes the "Topic B" a queue ;-) It is fine, but I am wondering if such a design would not "hurt" a Kafka-native ;-)
From my preliminary readings on Kafka, it is clear that I should also learn more about Storm... but, then, I have discovered Flink that, according to the graph at https://flink.apache.org/features.html, looks MUCH more performant than Storm, and now #morganw09dev has mentioned Samza! That means that I don't know where to start ;-)
Ultimately, I would like to code my transformations in XSLT, and, in the Java world, I think that Saxon is one of the leading XSLT processor. Do you know any "integration" of Saxon with Storm, Flink or Samza? Or, maybe my question does not make sense and I have to find another "way" to use Saxon with Kafka.
At the moment I am writing this comment, there are not (yet) any questions tagged with both [saxon] and any of the [apache-kafka], [apache-storm], [apache-flink] and/or [apache-samza].
Kafka itself cannot be used to transform data. It's only used for storing data to be consumed later.
One thought is having a three part architecture.
Kafka Topic A => Transformer => Kafka Topic B
Per your example. Your producer pushes COBOL related data to Kafka Topic A. Your Transformer reads from Topic A, does the necessary transformations and then outputs JSON to Topic B. Once in Topic B the front end application can then read it in its preferred format. If you go that route, the Transformer could be custom built using Kafka's default consumer and producer, or use a streaming framework such as Apache Samza or Apache Storm to help handle the messaging. Both Samza and Kafka were initially developed at LinkedIn and I believe work fairly naturally together. (Though I have never tried Samza).
I've spent the last few days looking around for an existing solution to a functional testing problem, but I am out of ideas and would appreciate some SO help!
I've got a preexisting suite of functional networking tests currently written in C++ using Boost.Test and Google Test, but might become rewritten into Rust soon. These generally take the following form:
unit test fixture {
1. Start a thread representing "the server" which goes and listens on some localhost port for incoming network connections.
2. Do client stuff representing "the client" to that localhost port.
3. Join the server thread, fetching an errors or problems.
4. Exit with success or failure.
}
This is great, and it works well. However it only tests loopback and in the real world the server component is probably in its own process running behind a NAT routed network, so it's not particularly realistic and therefore not really testing the code. What I think I'm looking for is some method of splitting the server thread part off into its own process, and then some method of getting the server test process and the client test process to work together to run the functional tests. Ideally speaking the server and client processes would run in separate "machines", this is something I can automate using OpenVZ scripting fairly easily so consider that problem out of scope, though it makes forking the process non-ideal.
I had been thinking that in this age of Web 2.0 et al surely this is a very common functional test scenario, and therefore that established patterns and test frameworks would abound. I mean, I as an old timer thinks "DCOM" as my first thought on how to solve this, though that's a 1990s Microsoft only solution. And maybe there is some modern and portable equivalent, and I am not searching for the right terms here, so here is my first question:
Is there any standard functional testing library or framework which extends Google Test or Boost.Test etc which lets you easily choose at runtime whether the server and client parts of each functional test should run as threads or as processes or best of all, as processes inside their own virtual machine with its own network stack?
This test scenario is surely common as muck. But let's assume it isn't, and no such tool exists. If it doesn't, then we need to extend Boost.Test or Google Test with some extra support. Firstly, we need to associate with each test fixture a supporting "server" part test fixture, and for the threaded test scenario we need to always run the server and client test fixtures concurrently. So, my second question:
Is there any way of strongly associating two test fixtures in any of the popular C++ or Rust unit testing frameworks where the two fixtures are seen as two halves of the same test, and always executed concurrently?
This leaves the second part: how to get a unit test framework to execute only the client parts in one process and only the server parts in the other process, and to do both always concurrently and in sync with one another, and moreover to merge the junit XML output from both parts into a single test result. So:
Is there any alternative functional testing approach, methodology, or open source solution which is better suited for distributed network functional testing than unit test frameworks such as Google Test or Boost.Test? Preferably something libvirt aware so it can orchestrate virtual machines as part of the testing setup and teardown? For example, is there some Jenkins plugin or something which could use Jenkins slaves in each OpenVZ container to orchestrate the concurrent execution of the multiple parts of each of the functional tests? Or is old fashioned CORBA still the least worst solution here? Is there maybe some way of automatically wrapping up test fixtures into a REST HTTP API?
I did do a quick review of the major integration testing frameworks, so Citrus, STAF and Twister. I'll be honest in saying they all seem way overkill for what I want which is a quick and easy way of making the existing functional test suite use a more realistic network routing than loopback. That's all I really want essentially, and I don't care how it's done so long as the check and requires still appear in Jenkins. Over to you Stackoverflow!
My thanks in advance for any help.
I have had similar requirements but I am come from the Java side of the world. What you can easily do is having a distributed management of nodes / machines using jGroups.
Once you understand how it works you can build a distributed system of nodes by just using 100 lines of code. With this system you can span and control child processes on each of those systems and check output and everything yourself. Should only cost you a day to take a jGroup example and get this running.
Once you have the infrastructure copy code and execute it as independent process your control is easy. Then use some of those nodes and get Selenium up and use a number of browser windows and execute scripts (or use Sikuli) and do your testing. Since the Selenium process is again Java you can generate all kind of reports you print to console or send directly to the cluster since those processes can join the cluster using jGroups too.
Such a system can be implemented in a week and it is really under your control. Very simple to do and very extendable.
Also you can provide plugins for Jenkins, Jira or Quality Center to interact with it and trigger test execution and management.
okay this might be a pretty lame and basic question but its stuck in my head since i never had chance to work on web-services.
We can get the same "text bases" response(xml, json etc) from our server by very basic/simple implementations (lets say servlet) then why do someone has to develop a web-service.
What is the exception thing a web-service gives over simple http response?
At a basic level, you are quite correct, from a low level point of view, it's just text (XML) on a socket.
For simple web services, a servlet is adequate (I'm writing one of these as we speak).
When talking about something like SOAP and WSS-* web services, however, there is a lot of boiler plate processing and features available from the standards that web service toolkits expose as higher level transactions.
A simple example is data marshaling. If you treat it purely as XML, then your service basically gets to process the XML by hand -- parse it, evaluate it, populate your internal models, etc.
Contrast this to something like this from Java EE:
#WebService
public Person getPerson(String personId) {
Person p;
...
return p;
}
The web service stack will convert your Person object in to a SOAP compliant XML blob. It will also produce a WSDL that you can use to create client code (on many platforms: .NET, PHP, etc.) to make the web service code.
In the end, your client and server have only a few lines of code, while the frameworks do all of the grunt work parsing, marshaling, and publishing for you.
So, the value of the WS stack is that it handles much of the bureaucracy of writing WSS compliant web services.
It's no panacea, but for many modern implementations, SOAP <-> SOAP remote processing can be a, mostly, cross platform, drag and drop affair.
It depends. If your web service needs to answer a simple yes/no question like "does this username exist?", then return yes, no, 0, 1, etc may be enough. If you have one that returns all the faculty attributes, XML or JSON may be appropriate because of the structured nature. It's a little less prone to parsing errors than trying to parse plain text.
Even though I've been in Java SE for quite some time now, I started EE & web w/ Java only about a month ago, so pardon if the question seems a bit noobish...
So here's the situation: I'm trying to write a JS based multi-player game with real-time interaction (let's say chess in this example, though it really doesn't matter what particular game it is, could be tennis or w/ever). The clients would interact with the server through JS calls, sending the move etc. Now, while I could just receive the move from one client & pass it straight on to the other player, not maintaining the game state on the server would mean putting a huge sign out saying "user JS scripts welcome" (and that's out of experience -- "hacked" a crapload of that kind myself). This brings me to my problem -- how do I share a stateful object between several sessions? One idea that came to mind was a singleton storing a Hashmap of stateful beans & then each session could retrieve the bean by it's hash, but I've no idea how right that is (and it seems rather complex for a fairly common thing like that). Tieing it to application scope seems overkill as well...
P.S. I do understand that the object would need concurrency managing etc, I just can't seem to put my finger on how to get it shared...
EDIT: I'm sorry I didn't mention it before -- using Glassfish, EE6.
You have a business process scenario which is defined according to Seam framework documentation as follows
The business process spans multiple interactions with multiple users, so this state is shared between multiple users, but in a well-defined manner. The current task determines the current business process instance, and the lifecycle of the business process is defined externally using a process definition language, so there are no special annotations for business process demarcation.
Here you can see a Seam business process management Tutorial
Notice Seam uses JBoss BPM behind the scenes to handle its business process context. If you just want to use plain JBoss BPM capabilities, you can see here how to integrate with JBoss
See also JBoss BPM User guide
Solved. Shared it via ServletContext, which I initially thought wouldn't work 'cause FacesServlet is a separate one, thought it has smthn like a different container either.