I am beginning to get into socket programming. Currently, I am transferring data between server and clients using scp which scales very poorly when dealing with streams of data (it seems like each new scp session needs to open a new TCP connection and this really slows down the speed).
I would like to transfer text to multiple clients, over a day, this text could reach a couple gigabytes in size so implementing some sort of compression is key.
Can anybody recommend some good libraries or wrappers which can simplify writing this code? The standard C++ sockets interface is quite cumbersome to work with. So far, my only lead is Boost ASIO but that doesn't seem to have compression capabilities. Any suggestions would be much appreciated.
For the compression part, you can use zlib. There are many C++ interfaces out there for zlib, or you can use it directly to compress and decompress messages.
try UDT.
UDT is a reliable UDP based application level data transport protocol for distributed data intensive applications over wide area high-speed networks. UDT uses UDP to transfer bulk data with its own reliability control and congestion control mechanisms.
I dont really know if the compression is available ...
Compression and multicast are two orthogonal issues. As said by previous posters, pick a compression library best suited for your data.
For multicast there are multiple options, OpenPGM and RSP are open source.
Related
I have a server which is able to relay the WebRTC media data from A to B. For the video conferencing, if we go with P2P approach then a mesh network is created. Whenever P2P doesn't work, we can have this relay server.
The main problem is that in the mesh network, the number of upload link is "N - 1" for N participants. Hence the number of connection goes upto N * (N-1). Usually mesh network allows 5-6 stable connections.
Many online sources suggest to implement SFU. If SFU decrypts the media data and then re-encrypts for every peer then that virtually requires a WebRTC component on the server side.
Are there any lightweight C/C++ based library which helps in this regard?
Any better alternative strategy for the same?
BTW, I tried to share the same offer to all the peers with their own answer, but as expected it didn't work. The peer gets disconnected after receiving few chunks.
I have referred below related posts:
WebRTC - scalable live stream broadcasting / multicasting
Multi-party WebRTC without SFU
Can I re-use an "offer" in WebRTC for mulitple connections?
There are quite a few free and open source projects that implement an SFU:
Jitsi is probably the best known, but it is written in Java, and might therefore be unsuitable in some deployments;
Janus is written in C; it is small, efficient, and well supported, but might not be the easiest to understand;
Ion-SFU and Galène are written in Go, and might be easier to adapt to your needs.
I am implementing a messaging system using C++ and Qt. After much thought, I have determined that multicasting or a multicast-style technique will work best to solve my problem. However, I have learned about UDP's unreliability and believe it to be unacceptable.
My requirements are as follows:
Messages are to be sent in a binary serialized form.
From any given node on the network, I must be able to send messages to the other nodes.
Message delivery must be insured.
I have heard of OpenPGM and NORM as alternatives for UDP. If anyone has experience with either of these, could you please share?
I am also open to the possibility of implementing "reliable" multicast by myself, in the application layer, but I would prefer not to if there is a library that already implements this.
I am using C++ and Qt, therefore .NET or Java-based solutions are not acceptable unless they are open-source and I may port them to C++.
Thank you very much.
EDIT 20120816T1853 MDT: An additional question: would either PGM or NORM have to be implemented at the hardware/IP level? Or can they be overlayed on top of existing protocols?
We have implemented our own reliable multicast protocol over UDP called RSP, since we needed something cross-platform and at the time couldn't find a good solution between Linux and Windows. The Windows PGM implementation disconnects slow clients which leave the send window, whereas our implementation throttles the sender similar to TCP. Afaik OpenPGM can be configured to do the same.
The open source NORM at http://cs.itd.nrl.navy.mil/work/norm can be built as a library and has C++ API with Python and Java language bindings. If you ping the developer (me) via the mailing list, I can help get you started.
There is a large RFC division of reliable multicast protocols, and many implementations out there. It's a long time since I looked at this but there are TRAM, LRMP, ...
In a few weeks I'm thinking of helping a project finish a pretty major aspect of a C++ world with 3D characters: networking. I will help with the server's transfer of information from/to clients. I already know C++ well enough. I just need to know what specifically I should know to do this and resources from which I could find this information. Thanks :)
as RageD said, it's a big difference in networking between different types of games. A FPS server typically sends complete game state to all clients regularly (e.g. 60Hz) over UDP. Other game types can use TCP (tuned a bit like TCP_NODELAY and forcing immediate ACK packets) or reliable UDP (raknet lib or others). Network protocol can become really wide so you'll need to think how to make it easily extendable. I'd recommend you to start from here: http://www.gamedev.net/community/forums/showfaq.asp?forum_id=15
So using any free opensource cross platform library like boost how to create a web service capable of reciving a data stream (for example stream of mp3 frames) on one URL like http://adress:port/service1/write/ and capable of sharing latest recived data to all consumers on http://adress:port/service1/read/ so of course mp3 is just an example of packed stream-able data - generally it can be anything packed. How to create such thing?
Generaly I am honesly triing to understend how to do such thing with C++ Network Library but it is just quite unclear to me.
The boost::asio documentation has four examples of complete HTTP server implementations, each with a slightly different threading architecture.
http://www.boost.org/doc/libs/1_43_0/doc/html/boost_asio/examples.html
You do not say what platform to use, but if Windows is an alternative, the Windows HTTP API easy to use and a great performer.
http://msdn.microsoft.com/en-us/library/aa364510(VS.85).aspx
I would like to create a chat application(desktop-app) in c++, so which protocol i would need to study and implement. UDP(?)
Please provide me some good thoughts and advices and links also.
UDP protocol is not the best choice for Internet chat program. UDP packets will be blocked by proxies. And UDP doesn't guarantee packets delivery. So probably TCP protocol will be a better choice.
Take a look on Boost.Asio Library. It already contains primitive implementation of chat program.
You don't give us much details here!
If your purpose is really to make a fully working and feature full chat application I suggest you look at XMPP which is an open instant-messenging protocol. Here is a list of some libraries implementing it.
If your purpose is to study network programming and you're more interested in UDP versus TCP for instance, then UDP is a bad choice for a chat application as it does not guarantee much about data integrity or ordering. Your messages might (and will!) be received in bad order or some might even be missing. TCP does that for kind of check for you.
In between (a very simple chat app) you can implement your very own protocol and use libraries others have suggested here like Boost.asio, ACE, POCO, or even wxWidgets and Qt, which will ease socket handling and also provide what you need to build a desktop app for the last 2.
Try using Boost.Asio. There are some examples of chat applications included in documentation.
You can use or look at an open-source networking library like ACE. A lot of goodies there.
You could use an existing library that handles instant messaging protocols, such as libpurple.
UDP is like a 'shoot and forget' kind of protocol. It's fast, but if you use it for communicating over the internet, there's no guarantee your messages will be recieved at all. Even if it's LAN, your packets can still be lost. It would be more convenient to use TCP which makes sure your packets arrive without errors and in the order you sent them.