I'm using Visual C++ 2013.
I have two applications (they run on same machine):
The first application receives high quality video streams from multiple clients, synchronizes these videos and writes them to video file (only H264, DIVX and MJPG)
The second application does some processing on each set of synchronized frames in these videos (using OpenCV)
However, I'm get stuck in exchange data between two processes since the read/write task to/from disk is quite slow. So I want to exchange data directly on RAM with low latency and process frame-by-frame.
I've read
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365574%28v=vs.85%29.aspx
but I have no idea dealing with this.
Please help me. Thanks a lot.
There are many ways of doing it. The most simple one is via WM_COPYDATA:
COPYDATASTRUCT cds;
cds.dwData = 0;
cds.cbData = 5 * sizeof(WCHAR);
cds.lpData = L"Hello";
::SendMessage(hTargetWnd, WM_COPYDATA, (WPARAM)GetSafeHwnd(), (LPARAM)&cds);
You can easily find your target window handle by calling:
HWND hTargetWnd = ::FindWindow(NULL, L"AppName");
The other method is based on shared memory. If you do use MFC framework there is CSharedFile class that implements this type of IPC.
You can also use UDP sockets for IPC. There are several benefits here:
There is no need to establish connection
The packet loss is almost impossible because the data is transferred via loopback interface.
Again if it is MFC-based app I would use CAsyncSocket class.
I guess that you have a stream of frames that you then distribute to the worker app for processing. I would consider using ZeroMQ for that. Reasons:
ZeroMQ is already message-based, so unlike e.g. TCP (stream-based) or UDP (limited frame size), you can transfer large messages without having to worry about fragmentation.
It manages things like (re-)connecting and some other things that reduce the complexity of your application.
It has a shared memory transport mechanism (you'd have to check if it is supported under MS Windows though) which should be efficient enough for your use cases.
It also supports TCP transport, in case you want to distribute the work to multiple machines.
It has bindings for many languages and for many OSs, in case you want to expand your scope later on.
Related
I have Visual C/C++ program which is going to communicate with other Programs(both local and Non-local) over the Intra-net(LAN).Previously I used win32 Memory Mapped Files/Events to communicate locally. Now I want to extend my program to support over network communication.
Since MMF doesn't support network Inter-process communication(term should be Intra-Process Com') it's obvious I have to use Win32 Named Pipes Or WinSock2 mechanisms in order to transferring data through local network.
Here my only intention to build a Intra-network Process communication only, So which mechanism perform better with respect to following factors:
Single Server Multiple Clients - (Duplex -> Transmit & Receive from both end)
Transmit data through slow Local Area Network
Server is basic Visual C/C++ Program and Client end is DLL loaded by another process therefore functionalities should be light-weight.
Transmission Speed (Data should transmit between process with higher frequency)
Which mechanism should be implemented ? Named-Pipes or WinScoket2 ?
Thank you.
I have an application that gets video samples from a frame grabber card via DirectShow. The application then does some processing and sends the video signal over a network. I now want to duplicate this video signal such that another DirectShow-enabled software (like Skype) can use the original input signal, too.
I know that you can create Tee filters in DirectShow like the one used to split a video signal for recording and preview. However, as I understand, this filter is only useful within a single graph, ie I cannot use it to forward the video from my process to eg Skype.
I also know that I could write my own video source, but this would run in the process of the consuming application. The problem is that I cannot put the logic of my original application in such a video source filter.
The only solution I could think of is my application writing the frames to a shared memory block and a video source filter reading it from there. Synchronisation would be done using a shared mutex or so. Could that work? I specifically do not like the synchronisation part?
And more importantly, is there a better solution to solve this problem?
The APIs work as you identified: a video capture application, such as Skype, is requesting video stream without interprocess communication in mind, there is no IPC involved to consume output generated in another process. Your challenge here is to provide this IPC yourself so that one application is generating the data, and then another extends existing API (virtual video source device) and picks existing data, then delivers as generated.
With video, you have a relatively big stream of data and you are interested in avoiding its excessive copying. File mappings (AKA shared memory) are the right thing to do: you put bytes in one process and they are immediately visible in another. You can synchronize access to the data using names events and mutexes which both processes use collaboratively - to signal availability of new buffer of data, as indication that used buffer is no longer in use etc.
I am new to creating Windows applications in C++. My task is to write two cpp files, one of which will send a number (x) to the other one, the other one will evaluate f(x) and send it back to the first one. I should implement it using Messages. Couldn't get anything specific online, Could someone pls give me a clue, where to start?
Great thanx!
Are you talking about window messages? If so, the sending app could use SendMessage, which would cause the receiving app to get its window procedure executed. Of course, this means that the receiving app needs to create a window whose window handle is somehow made available to the sending app.
You can do it in several ways.
Using WM_COPYDATA message to pass the data
Allocating global memory to pass data and sending your own message, such that second program can read the data from memory
Sending a message (if two ints suit your needs to pass data)
Using named pipes
Using TCP/IP local connection (peer to peer or through a server)
Look at ZeroMQ (http://zeromq.org ; cross-platform, LGPL). It is a very simple, lightweight and powerfull library. From the very basic level you can use it to exchange UDP-style datagrams, but through reliable transport (TCP or some variants). Also you have cancelling support, time-based polling and advanced network schemes (which are non-needed in your case). I've selected it for a similar task, and it performs very well.
I would like to write a program and run it on two machines, and send some data from one machine to another in an Ethernet frame.
Typically application data is at layer 7 of the OSI model, is there anything like a kernel restriction or API restriction, that would stop me from writing a program in which I can specify a destination MAC address and have some data sent to that MAC as the Ethernet payload? Then write a program to listen for incoming frames and grab the frames from a specified source MAC address, extracting the payload of data from the frame?
(So I don't want any other overhead like IP or TCP/UDP headers, I don't want to go higher than layer 2).
Can this be done in C++, or must all communication happen at the IP layer, and can this be done on Ubuntu? Extra love for pointing or providing examples! :D
My problem is obviously I'm new to network programming in c++ and as far as I know, if I want to communicate across a network I have to use a socket() call or similar, which works at an IP layer, so can I write a c++ program to work at OSI layer 2, are there APIs for this, does the Linux kernel even allow this?
As you already mentioned sockets, probably you would just like to use a raw socket. Maybe this page with C example code is of some help.
In case you are looking for an idea for a program only using Ethernet while still being useful:
Wake on LAN in it's original form is quite simple. Note however that most current implementations actually send UDP packets (exploiting that the receiver does not parse for packet headers etc. but just a string in the packet's payload).
Also the use of raw sockets is usually restricted to privileged users. You might need to either
call your program as root
or have it owned by root and setuid bit set
or set the capability for creating raw socket using setcap CAP_NET_RAW+ep /path/to/your/program-file
The last option gives more fine grained privileges (just raw sockets, not write access to your whole file system etc.) than the other two. It is still less widely known however, since it is "only" supported from kernel 2.6.24 on (which came with Ubuntu 8.04).
Yes, actually linux has a very nice feature that makes it easy to deal with layer 2 packets. You can use a TAP device, which allows your userspace program to read/write ethernet traffic through the kernel.
http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/tuntap.txt
http://en.wikipedia.org/wiki/TUN/TAP
I'm having problems with low performance using a Windows named pipe. The throughput drops off rapidly as the network latency increases. There is a roughly linear relationship between messages sent per second and round trip time. It seems that the client must ack each message before the server will send the next one. This leads to very poor performance, I can only send 5 (~100 byte) messages per second over a link with an RTT of 200 ms.
The pipe is asynchronous, using multiple overlapped write operations (and multiple overlapped reads at the client end), but this is not improving throughput. Is it possible to send messages in parallel over a named pipe? The pipe is created using PIPE_TYPE_MESSAGE, would PIPE_READMODE_BYTE work better? Is there any other way I can improve performance?
This is a deployed solution, so I can't simply replace the pipe with a socket connection (I've read that Windows named pipe aren't recommended for use over a WAN, and I'm wondering if this is why). I'd be grateful for any help with this matter.
We found that Named Pipes had poor performance from Windows XP onwards.
I don't have a solution for you. But I am concurring with the notion of Named Pipes being useless from XP onwards. We changed our software (in terms of IPC) completely because of it.
Is your comms factored into a separate DLL? Perhaps you could replace the DLL with an interface that looks the same but behaves differently?
I've implemented a work around, introducing a small (~1ms) fixed delay to buffer up as much data as possible before writing to the pipe. Over a network link with a RTT of 200ms, I can send ten times as much data in about a third of the time.
I send a message down the pipe when it first connects, so the client can determine the comms mode supported by the server and send data accordingly.
I would imagine that some of the WAN optimisation gear out there would be able to boost performance, as one of the things they do is understand protocols and reduce their chattiness. Given the latency of many WAN links, this alone can boost throughput and reduce timeouts.