boost.asio transfer files - not completely transferd? - memory mapping performance? - c++

Question

I am currently trying to implement a file-transfer app under linux using boost.asio. I am complete new to this topic (general learning cpp), the past days I was trying to figure out how this might work. I am already losing my mind.
I made some progress, but I can't transfer a file completely, instead I am just getting a part of the file. Does anyone knows why the buffer is not red or written completely?
I made It really simple, its just a series of commands, I will implement it object oriented later on.
The secondly I was wondering if there is another way to map the file in memory more efficiency? Say someone want to transfer a 2 tb file?
I am using this binary file for testing: blah.bin
to successfully build it u need:
g++ -std=c++17 -Wall -Wextra -g -Iinclude -Llib src/main.cpp -o bin/main -lboost_system -lpthread
server
//server
#include <boost/asio.hpp>
#include <iostream>
#include <fstream>
using namespace boost::asio;
using ip::tcp;
using std::string;
using std::cout;
using std::endl;
int main() {
boost::asio::io_service io_service;
//listen
tcp::acceptor acceptor_(io_service, tcp::endpoint(tcp::v4(), 3333));
//socket
tcp::socket socket_(io_service);
//waiting
acceptor_.accept(socket_);
//read
boost::asio::streambuf buf;
boost::asio::read_until(socket_, buf, "\nend\n");
auto data = boost::asio::buffer_cast<const char*>(buf.data());
std::ofstream file("transferd.bin");
cout << data;
file << data;
file.close();
//response
boost::asio::write(socket_, boost::asio::buffer("data recived"));
return 0;
}
client
//client
#include <boost/asio.hpp>
#include <iostream>
#include <fstream>
#include <vector>
using namespace boost::asio;
using ip::tcp;
using std::string;
using std::cout;
using std::endl;
using std::vector;
const vector<char> fileVec(const std::string & fileName) {
std::ifstream file(fileName, std::ios::in | std::ios::binary);
vector<char> tempVec ((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
file.close();
return tempVec;
};
int main() {
boost::asio::io_service io_service;
//socket
tcp::socket socket(io_service);
//connection
socket.connect(tcp::endpoint(boost::asio::ip::address::from_string("127.0.0.1"), 3333));
//write to server
auto vdata = fileVec("example.bin");
vdata.push_back('\n');
vdata.push_back('e');
vdata.push_back('n');
vdata.push_back('d');
vdata.push_back('\n');
boost::system::error_code error;
boost::asio::write(socket, boost::asio::buffer(vdata), error);
//response from server
boost::asio::streambuf receive_buffer;
boost::asio::read(socket, receive_buffer, boost::asio::transfer_all(), error);
const char* response = boost::asio::buffer_cast<const char*>(receive_buffer.data());
cout << response;
return 0;
}

Answer 1

The problem is not in the socket but how you are writing the file in the server.
std::ofstream file("transferd.bin");
cout << data; // you cannot print binary data like this on the standard output!
file << data;
file.close();
The above snippet is wrong because the << operator is used for ASCII not for binary data!
A simple fix would be to replace it with the following snippet:
std::ofstream file("transferd.bin");
file.write(data, buf.size());
The second part of the question is of course more hard and it requires a lot of code changing.
The point is that you cannot transfer all the content at once, But you should split the transfer in small chunks.
One solution can be to send a small header with some information like the total transfer bytes so the server can read chunk by chunk until the whole transfer is complete.
The message has a header file containing the total message size, the number of chunks. Each chunks have a little header indicating the chunk size or for instance the chunk index in case you wanna switch to UDP.
Following the server snippet
#include <array>
#include <boost/asio.hpp>
#include <cstddef>
#include <fstream>
#include <iostream>
#include <vector>
using namespace boost::asio;
using ip::tcp;
using std::cout;
using std::endl;
using std::string;
struct MessageHeader {
int64_t totalSize;
int64_t chunkCount;
};
struct ChunkHeader {
int64_t index;
int64_t size;
};
MessageHeader parseHeader(const char* data) {
MessageHeader header;
memcpy(&header, data, sizeof(MessageHeader));
return header;
}
ChunkHeader parseChunkHeader(const char* data) {
ChunkHeader header;
memcpy(&header, data, sizeof(MessageHeader));
return header;
}
MessageHeader readHeader(tcp::socket& socket) {
std::array<char, sizeof(MessageHeader)> buffer;
boost::asio::read(socket, boost::asio::buffer(buffer));
return parseHeader(buffer.data());
}
ChunkHeader readChunkHeader(tcp::socket& socket) {
std::array<char, sizeof(ChunkHeader)> buffer;
boost::asio::read(socket, boost::asio::buffer(buffer));
return parseChunkHeader(buffer.data());
}
std::vector<char> readChunkMessage(tcp::socket& socket) {
auto chunkHeader = readChunkHeader(socket);
std::vector<char> chunk(chunkHeader.size);
boost::asio::read(socket, boost::asio::buffer(chunk));
return chunk;
}
int main() {
boost::asio::io_service io_service;
// listen
tcp::acceptor acceptor_(io_service, tcp::endpoint(tcp::v4(), 3333));
// socket
tcp::socket socket_(io_service);
// waiting
acceptor_.accept(socket_);
auto messageHeader = readHeader(socket_);
for (auto chunkIndex = 0ll; chunkIndex != messageHeader.chunkCount; ++chunkIndex) {
auto chunk = readChunkMessage(socket_);
// open the file in append mode
std::ofstream file("transferd.bin", std::ofstream::app);
file.write(chunk.data(), chunk.size());
}
// response
boost::asio::write(socket_, boost::asio::buffer("data recived"));
return 0;
}
The above solution has drawbacks because everything is synchronous and if the client quit in the middle of transfer the server will be stuck :D
A better solution is to turn that in async code... but It's too much all at once for a beginner!