I'm right now I have a Thrift TSimpleServer that is going to accept connection from multiple clients at once. Once those clients connect they will be sending data that is to be stored in a respective file on the server. Now I'm pretty new to thrift so I'm not sure how the TSimpleServer handles multiple connections(haven't tested that yet) anyways I'm not even sure how to go about saving the received data in separate files simultaneously. I assume a separate thread is created to handle each client connections but how do give each thread a file path to use?
TSimpleServer handles simply :-) one connection at a time. You may want to use TThreadedServer or TThreadPoolServer which are able to process requests in parallel.
How you do give each thread a file depends on your exact use case. For example, you could consider deriving an own version of a thread factory and register this one with the thread Manager. Although the following blog post does not exactly cover your scenario, you should get the idea from the code at the end of the text:
http://mikecvet.wordpress.com/2010/05/13/apache-thrift-tutorial-the-sequel/
Related
I have written a c++ app on Windows that uses a secondary thread to communicate with a PLC. The content of the messages are vectors (of fixed size around 120) of uint_16t. The interface also allows to send something to the PLC.
I am really new to this so I want to start correctly. I would like now to separate the communication from the app and run it in a separate process. One reason for that is that I would like to write other apps that could also use this process for communicating with the PLC. The communicating process would thus organise and distribute the messages, since all app don't share the same messages from or to the PLC. So I still need to receive the vector of uint_16t (and for that I need to let know to the communicating process which one to read!) but also to send some things to the PLC. Thus I would need a bidirectional communication.
I have seen that there are many possibilities to achieve this but I would like to communicate via Message Queues (Message Passing). If possible, I would also like to allow apps on other computers to connect to this communicating process.
I am therefore looking for a c++ library to do that. I would appreciate any advises about which one to favor.
I can use message queue from Boost but it doesn't allow to communicate with another computer.
As a secondary question related to this one, I would like to start the communicating process if at least one interface process is opened. How to achieve that synchronization?
I thank you in advance for your help.
I have a piece of software in QT framework (c++) that's suppose to dispatch processed (local) data to other servers and receive the same (foreign) data processed on other servers and compare it.
Problem occurs when a large amount of local data is processed foreign data is buffered and doesn't go into comparison process until all local data is sent. I need the data to be compared in certain time frame, so this causes a timeout.
An idea was to to use one thread to dispatch local data and another thread to receive and compare foreign data. QTcpServer will probably need a mutex to protect it from simultaneous reading and writing.
Is this possible to do with one connection or would it be better to have one connection for dispatching and one for receiving in QT environment?
I checked the Fortune server example
http://doc.qt.io/qt-5/qtnetwork-threadedfortuneserver-example.html
but I need to know if it's possible and logical to use different threads for sending and receiving on the same connection.
PS. I'm new to multi-threading so I apologise if I misunderstood some concepts.
Without seeing any code, it's difficult to definitively answer this question. However, this may set you on the right track...
I wouldn't expect you'd need different threads for sending / receiving data; QTcpSocket is asynchronous.
It sounds like the architecture you're using to process the data may need revising.
foreign data is buffered and doesn't go into comparison process until all local data is sent
That sounds like more of an issue and the area where multi-threading would be beneficial. So, use multi-threading for processing the data, rather than controlling the communication between servers.
As you state you're new to multi-threading, I suggest starting by reading this article and using its examples as a template.
I am new to quickfix (I'm a student trying to teach myself), and have downloaded the examples from quickfix.org (in c++) and have been able to connect ordermatch to tradeclient and get them talking to each other. I changed the config file for ordermatch to allow multiple clients and got that working (ordermatch can receive orders from multiple clients and manage the order book).
I have been trying to find a way to alter ordermatch to send it's confirm messages to ALL clients, not just the sender.
I have a seperate implementation of a limit orderbook and want to crack the incoming messages (orders, cancels, etc) and store them in my limit orderbook. My orderbook watches the book an makes trading decisions based on it. The problem is, I can't figure out how to get ordermatch to send all updates to this client. Further, I am having a hard time figuring out how to "soup up" the tradeclient to not only send orders, but receive and crack them.
I'm thinking I need to have an acceptor and an initator in each application(in ordermatch and in one of the tradeclients)--I've read this is possible and common but can't find any sample code. Am I on the right track here, or is there a better way to set this up? Does anybody have some sample code they can share? I am not planning on using this for live trading so crude code is perfectly fine by me.
Thanks in advance
Brandon
Same application can act as Initiator for one session and Acceptor for different session.
Infact you can have multiple Acceptor/Initiator sessions from same application.
Config file needs to define multiple sessions.
Or you can have separate config file for each session.
If I understand correctly, I think what you're trying to do is intercept messages between an OMS and a broker (c.f. client and server) and act depending on what they contain. There are a few ways you could do this, including intercepting at the TCP layer, but I think that the easiest way might be to use 2 separate programs as #DumbCoder suggests and connect to one of them as an acceptor from your clients, process the messages and then pass them on to another program via another protocol and then send them on from the other program. Theoretically you can create another instance of the engine in your program and, by using different config files on creation (when FIX::FileStoreFactory storeFactory(*settings); is called) of each instance of the engine. However, I have never seen this done and so feel that it could cause problems. If you do try this method I would strongly advise putting the initiator and the connector in different dlls which might just separate the two engine instances enough.
I have a simple c++ application that generates reports on the back end of my web app (simple LAMP setup). The problem is the back end loads a data file that takes about 1.5GB in memory. This won't scale very well if multiple users are running it simultaneously, so my thought is to split into several programs :
Program A is the main executable that is always running on the server, and always has the data loaded, and can actually run reports.
Program B is spawned from php, and makes a simple request to program A to get the info it needs, and returns the data.
So my questions are these:
What is a good mechanism for B to ask A to do something?
How should it work when A has nothing to do? I don't really want to be polling for tasks or otherwise spinning my tires.
Use a named mutex/event, basically what this does is allows one thread (process A in your case) to sit there hanging out waiting. Then process B comes along, needing something done, and signals the mutex/event this wakes up process A, and you proceed.
If you are on Microsoft :
Mutex, Event
Ipc on linux works differently, but has the same capability:
Linux Stuff
Or alternatively, for the c++ portion you can use one of the boost IPC libraries, which are multi-platform. I'm not sure what PHP has available, but it will no doubt have something equivalent.
Use TCP sockets running on localhost.
Make the C++ application a daemon.
The PHP front-end creates a persistent connection to the daemon. pfsockopen
When a request is made, the PHP sends a request to the daemon which then processes and sends it all back. PHP Sockets C++ Sockets
EDIT
Added some links for reference. I might have some really bad C code that uses sockets of interprocess communication somewhere, but nothing handy.
IPC is easy on C++, just call the POSIX C API.
But what you're asking would be much better served by a queue manager. Make the background daemon wait for a message on the queue, and the frontend PHP just add there the specifications of the task it wants processed. Some queue managers allow the result of the task to be added to the same object, or you can define a new queue for the finish messages.
One of the best known high-performance queue manager is RabbitMQ. Another one very easy to use is MemcacheQ.
Or, you could just add a table to MySQL for tasks, the background process just queries periodically for unfinished ones. This works and can be very reliable (sometimes called Ghetto queues), but break down at high tasks/second.
i want to develop a pretty basic client-server program.
one software reads xml (or any data) and send it to the server who in turn will manipulate it a little bit and eventually will write it to the disk.
the thing is that if i have many xml files on disk (on my client side), i want to open multiple connection to the server , and not doint one by one.
my first question is : let's say i have one thread who keeps all the files handles and waitformultipleobjects on them, so it will know when one of them is ready to be read from disk. and for every file i have an appropriate socket who suppose to send that specifi file to the server. for the socket i can use the select function to know which sockets are ready for sent. but is there way to know that both the file and the appropraite socket are ready to be sent ?
second, is there a more efficient way to design the client, cuase on my current design i'm using just one thread which on multi processor computer is rather not efficient enough.
(though i'm sure is till better then laucning new thread for every socket connection)
third, for the server i read about the reactor pattern. it seems appropriate but still ,like my second question, seems not effient enought while using one thread.
maybe i can use something with completion ports ? think they are pretty efficient but never really used them, so don't know exactly how.
any answers and general suggestion would be great.
Take a look at boost::asio it uses a proactor pattern (see the docs) that basically uses the OS wait operations (waitforsingle/multiple,select,epoll, etc...) to make very efficient use of a single thread in a system like you're looking at implementing.
asio can read/write files as well as sockets. You could sumbit an async read for the file using asio, it would call your callback on completion then you would submit that read buffer as an async write to the socket. Asio would take care of delivering all async writes buffers as the socket completed each pending write operation.
Each of these operations is done asynchronously so the thread is only really busy to initiate reads or writes, sitting idle the rest of the time.