I am coding network stuff via tcp/ip.
Specifically I have been using boost::asio.
Recently, to ease coding, I started using boost::asio::ip::tcp::iostream.
It can be useful for fast developing! But I am not sure whether it uses async_read or async_write.
Does anybody know whether it does?
No it cannot. That's most of what makes it so simple.
However, you can force a full-duplex experience using boost::asio::ip::tcp::iostream with some tweaks. You'll need to run the reading/writing tasks on separate threads to achieve the parallellism without having asynchrony.
An examples of this - what I consider to be a - hack, is here:
How to avoid data race with `asio::ip::tcp::iostream`?
That answer does also give the asynchronous approach, so that may help remove the intimidation factor because you can simply compare side-by-side both approaches.
Related
There are 2 c++ applications where one application let say A is reading from an interface device and does some processing and need to provide the data in certain format to an application B.
I feel this can be done in 2 ways as mentioned below -
1. I serialize the data structure in app A and write it to a socket.
2. I inject the packet to an interface.
Please help to evaluate which option would be faster. Or if there is another way to do it faster.
I'm not sure what you mean by "I inject the packet to an interface."
Anyway, if your 2 applications are or could be on separate machines, go for the socket solution.
If on the same machine, you can implement some type of interprocess communication. I recommend you to use Boost for this: http://www.boost.org/doc/libs/1_56_0/doc/html/interprocess.html
As far as performance is concern, ideally you want to perform some tests to find out which work better in your scenario. Also, if you're already familiar with sockets, it may be simpler to use them.
I am working on a project which involves several C++ programs that each take input and generate output. The data (tens to hundreds of bytes, probably JSON) essentially flows (asynchronously) in one direction, and the programs will need to be located on different Linux computers around the LAN.
Since the data flows in only one direction, I don't believe I need a transactional model like HTTP. I think a message queue model (fire and forget) makes the most sense and should simplify the logic of each program. It is probably sufficient to merely note that the message was added to the remote queue successfully.
What I am looking for are recommendations for how to implement this message queue in C or C++. It seems like POSIX and Boost message queues are limited to a single host, and RabbitMQ seems to have weak C/C++ support, and MQ4CPP seems inadequately supported for a business-critical role. Am I wrong about this? What about Boost ASIO or ACE or writing socket code myself? I look forward to your suggestions.
In terms of simple messaging support, ZeroMQ is hard to beat. It's available in many language bindings and supports everything from simple send and receive to pub/sub, fanout, or even a messaging pipeline. The code is also easy to digest and makes it pretty easy to switch between patterns.
Looking at their Weather Update Server sample (in 20 some odd languages) shows how easy it can be to create publish/subscribe setups:
zmq::context_t context (1);
zmq::socket_t publisher (context, ZMQ_PUB);
publisher.bind("tcp://*:5556");
publisher.bind("ipc://weather.ipc");
while(1) {
// Send message to all subscribers
zmq::message_t message(20);
snprintf ((char *) message.data(), 20 ,
"%05d %d %d", zipcode, temperature, relhumidity);
publisher.send(message);
}
I've used it on some mixed C# and Python processes without much hassle.
Personally, if I understand the question, I think that you should use a lower-level TCP connection. It has all of the guarantied delivery that you want, and has a rather good Berkley Sockets API.
I've found that if your willing to implement a very simple protocol (eg. four-byte NBO message length, n bytes of data), you can get very simple, very customizable, and very simple. If you go with this, you also (as mentioned) get great C support (which means C++ support, although things aren't in classes and methods). The socket code is also very easy, and they have asynchronous IO with the standard async flags for the Linux/UNIX/POSIX IO functions (thats one of the other benefits, if you know anything about POSIX programing, you basically know the socket API).
One of the best resources for learning the socket API are:
Beej's Guide to Network Programing: http://beej.us/guide/bgnet/, this is very good if you need the overall programming model in addition to specifics
Man Pages: If you just need function signatures, return values, and arguments, these are all you need. I find the Linux ones to be very well written and useful (Proof: Look at my console: man, man, man, man, man, make, man, ...)
Also, for making data network-sendable, if your data is JSON, you have no worries. Because JSON is just ASCII (or UTF-8), it can be sent raw over the network with only a length header. Unless your trying to send something complicated in binary, this should be perfect (if you need complicated in binary, either look at serialization or prepare for a lot of Segmentation Fault).
Also, you probably, if you go the socket path, want to use TCP. Although UDP will give you the one-way aspect, the fact that making it reliable is pitting your home-baked solution against the top-of-the-line TCP given by the Linux kernel, TCP is an obvious option.
RabbitMQ is just one implementation of AMQP. You might want to investigate Apache Qpid or other variants that might be more C/C++ friendly. There is a libamqp for C though I have no first hand experience with it. I don't know exactly what your requirements are but AMQP, properly implemented, is industrial strength and should be orders of magnitude faster and more stable than anything you are going to build by hand in a short amount of time.
I am using Boost Serialization and socket sending for a similar application. You can find an example of serialization here :
http://code.google.com/p/cloudobserver/wiki/TutoriaslBoostSerialization
And on this page:
http://www.boost.org/doc/libs/1_38_0/doc/html/boost_asio/examples.html
under serialization you will find examples on how to make servers and clients. Make one server on a particular port and you can generate multiple clients on multiple computers which can communicate with that port.
The downside to using boost serialization is that it has a large overhead if you have a simple data structure to be serialized but it does make it easy.
Another recommendation is the distributed framework OpenCL. The document The OpenCL C++ Wrapper for API provides further information on the library. In particular, the API function cl::CommandQueue could be of interest for creating queues on devices within a network setup.
Another messaging solution is ICE (http://www.zeroc.com/). It is multi-platform, multi-language. It uses more of an RPC approach.
I'm currently programming a chat system based on a server - client model and using TCP as the communication protocol. Although it's working as expected, I'd like to further optimize important parts on the server side.
The server uses four extra threads to handle new connections, console input, etc, without blocking normal chat conversations. Well, there is only one thread for all messages that are being sent from client to client, so I assume it would be good to optimize the code there, as it would be the most obvious bottleneck. After reading the data on each client's socket, the data has to be processed using different steps. One of those steps would be to check for blocked words. And that's where my original question starts.
I played with std::string::find() and the strstr() function. According to my tests, std::string::find() was clearly faster than the old C-style strstr() function.
I know that the std::string is very well optimized, but C-style char arrays and their own functions always seemed to be somewhat faster, especially if the string has to be constructed over and over again.
So, is there anything faster than std::string::find() to scan a series of characters for blocked words? Is std::string::find() faster than strstr(), or are my benchmarks lousy? I know that the gain may be negligigle compared to effort needed to keep C-style char arrays and their functions clean, but I'd like to keep it as fast as possible, even if it is just for testing purposes.
EDIT: Sorry, forgot to mention that I am using MSVC++2010 Express. I am only targeting Windows machines.
Have you benchmarked to verify that lots of time is in fact being taken in the check for blocked words? My completely naive guess is you're gonna be spending lots more time waiting for RPCs than any local processing...
Have you tried the regular expressions library in either C++11 if you use that, or Boost if you don't? I'm not sure about the speed, but I believe they perform quite well. Additionally, if you are using this as a form of profanity filter, you'd want regular expressions anyway to prevent trivial circumvention.
There exist faster searching-algorithms than the linear search typically used in STL, or strstr.
Boyer-Moore is quite popular. It requires preprocessing of the target-string, which should be feasible for your usecase.
Exact string matching algorithms is a free e-book with an in-depth description of different search-algorithms and their tradeofs.
Implementing more advanced algorithms could take considerable effort.
As said in the other answers, It is doubtful that string-searching is a bottle-neck in your chat-server.
I'm fairly new to advanced C++ program techniques such as templates,
but I am developing a simple API for a project I'm working on.
The function or method that you call can take a long time to complete.
Essentially it's transferring a file over the network.
It looks a bit like this.
Client
{
int WriteFile();
int ReadFile();
}
But I want to have a couple of options here.
call WriteFile and have it block.
Call WriteFileAsync and not have it block.
In the async version be flexible about how I know the task is done.
Be able to poll the client to find out where it's up to with my current Read or Write
operation.
I'm at a bit of a loss as to how to design this nicely the C++ way.
It's a requirement to avoid using boost, but I could use a boost-like approach.
Although, I looked through some of the headers and got very much confused. Anything beyond
basic template programming for me I find confusing.
What I'm after is a nice way of being notified of event completion and be able to wait for
an event to complete.
My advice would be looking at the docs and tutorial for boost::asio (which you can use as part of boost or as part of the independent asio project, but I guess that the requirement is no external libs, not just no boost).
Usually blocking calls are simple to define, while non-blocking operations require some callback mechanism as to notify the user of the result of the operation whenever that completes. Again, take a look at the tutorials and docs to get an idea of a clean interface, that will be much easier to browse over than the headers.
EDIT: ASIO has support for different protocols, so it might be more complex than what you need, read one of the examples and get the ideas of how to use callback mechanisms.
Regarding the use of asynchronous calls, I would suggest reading about the design of the future for C++0x.
Basically, the idea is to hand a proxy to the user, instead of a plain type. This proxy is aware of the threading and can be used to:
poll about the completion
get the result
You can also add clever mechanisms like trying to get the result for a fixed duration or up to a fixed point in time and abandon (for the moment) if the task hasn't completed in time (for example to do something else and try again later, or to simple go forward and forget about this).
The new threading API of C++0x has been very cleverly designed (taking mainly after Boost.Threads) and would give you much insight as to how to design for multi-threading.
I've created a model for executing worker tasks in a server application using a thread pool associated with an IO completion port such as shown in the posts below:
http://weblogs.asp.net/kennykerr/archive/2008/01/03/parallel-programming-with-c-part-4-i-o-completion-ports.aspx
http://blogs.msdn.com/larryosterman/archive/2004/03/29/101329.aspx
Are there any classes in boost that can help with this programming model?
Not really, at least, not last time I looked. I mean, boost::thread_group might make things marginally tidier in places, but not so as would make much of a difference, I don't think.
Boost's thread support seems marginally useful when writing something that's cross-platform, but given that what you're writing is going to be Win32-specific anyway (due to the use of IOCPs) there doesn't really seem to be much benefit from that.
You might want to check out the threadpool project, which looks like a nice threadpool implementation on top of boost. I haven't tried it myself, but it looks fairly nice.
I haven't seen anything in boost that helps with the structure that you tend to end up with when using IO Completion Ports, but then I haven't looked that recently... However, slightly off-topic, you might like to take a look at the IOCP based thread pool that is part of my free IOCP server framework. It might give you some ideas if nothing else. You can find the code here. The thread pool supports expansion and contraction based on demand and has been in use in production systems for over 6 years.
ACE has some reactors that you can use to model things around your IOCPs. Some of these could have been added to boost, but boost makes building them pretty easy.