I am developing an encrypted version of a realtime communication application. The issue I have is, that the encrypted data pakets sent to the receiver are faulty. An example from the error log: (hex encoded data, the original data is pure byte code).
sent: 262C1688215232656B5235B691826A21C51D37A99413050BAEADB81D8892493FC0DB519250199F5BE73E18F2703946593C4F6CEA396A168B3313FA689DE84F380606ED3C322F2ADFC561B9F1571E29DF5870B59D2FCF497E01D9CD5DFCED743559C3EE5B00678966C8D73EA3A5CD810BB848309CDF0F955F949FDBA618C401DA70A10C36063261C5DBAB0FC0F1
received: 262C1688215232656B5235B691826A21C51D37A99413050BAEADB81D8892493FC0DB519250199F5BE73E18F2703946593C4F6CEA396A168B3313FA689DE84F380606ED3C322F2ADFC561B9F1571E29DF5870B59D2FCF497E01D9CD5DFCED743559C3EE5B00CDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCDCD
This is the call of the send-method:
string encSendBuffer = sj->cipherAgent->encrypt(sj->dFC->sendBuffer, sj->dFC->sendBytes);
char* newSendBuffer = new char[encSendBuffer.length() + 1];
strcpy(newSendBuffer, encSendBuffer.c_str());
sj->dFC->s->async_send_to(boost::asio::buffer(newSendBuffer, encSendBuffer.length()),
*sj->dFC->f,
boost::bind(&sender::sendHandler, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred)
)
sj->dFC->s is a UDP-Socket and sj->dFC->f is an UDP Endpoint.
The error code of the sendHandler is always system: 0
This is how I do the encryption using the Crypto++ library: (extract)
string cipherEngine::encrypt(char* input, int length)
{
string cipher = "";
CTR_Mode<AES>::Encryption e;
e.SetKeyWithIV(key, keyLength, iv);
ArraySource as((byte*)input, length, true,
new StreamTransformationFilter(e,
new StringSink(cipher)
)
);
return cipher;
}
UPDATE: Code of the receive function:
void receiver::receive(){
int maxLength = 4096;
sj->dFC->s->async_receive_from(boost::asio::buffer(input,maxLength),
senderEndpoint,
boost::bind(&receiver::handleReceiveFrom, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
}
After the Data is received, it is stored in the char buffer input and decrypted in the handleReceiveFrom function.
Without encryption everything is fine. The number of bytes that are sended is always correct, on receiver side too. The length of de "CD"- blocks are quite random. I already checked the encryption and the decrypted data is the same as the original plain text.
Does any know where this behavior comes from?
The key here is that the erroneous data begins after the first null (0x00) value in your encrypted data array. The following line:
strcpy(newSendBuffer, encSendBuffer.c_str());
...looks like it's only copying up to the data until that null byte into newSendBuffer. The send function is sending that buffer contents just fine; the buffer just doesn't have the data you expect. You'll need to load newSendBuffer in a different way, not using strcpy(), that can handle null bytes. Try std::memcpy().
Thank you Joachim Pileborg and Jack O'Reilly! You are right indeed.
I changed my code from strcpy(newSendBuffer, encSendBuffer.c_str());
to
for (int i = 0; i < encSendBuffer.length(); i++)
{
newSendBuffer[i] = encSendBuffer.at(i);
}
on sender and receiver side. It actually solved the problem. It is quite naive code but it does what it should.
std::memcpy() seems to be much more elegant and i will try it out.
Related
I'm using nanopb in a project on ESP32, in platformIO. It's an arduino flavored C++ codebase.
I'm using some protobufs to encode data for transfer. And I've set up the memory that the protobufs will use at the root level to avoid re-allocating the memory every time a message is sent.
// variables to store the buffer/stream the data will render into...
uint8_t buffer[MESSAGE_BUFFER_SIZE];
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
// object to hold the data on its way into the encode action...
TestMessage abCounts = TestMessage_init_zero;
Then I've got my function that encodes data into this stream via protobufs (using nanoPB)...
void encodeABCounts(int32_t button_a, int32_t button_b, String message)
{
// populate our data structure...
abCounts.a_count = button_a;
abCounts.b_count = button_b;
strcpy(abCounts.message, message.c_str());
// encode the data!
bool status = pb_encode(&stream, TestMessage_fields, &abCounts);
if (!status)
{
Serial.println("Failed to encode");
return;
}
// and here's some debug code I'll discuss below....
Serial.print("Message Length: ");
Serial.println(stream.bytes_written);
for (int i = 0; i < stream.bytes_written; i++)
{
Serial.printf("%02X", buffer[i]);
}
Serial.println("");
}
Ok. So the first time this encode action occurs this is the data I get in the serial monitor...
Message Length: 14
Message: 080110001A087370656369616C41
And that's great - everything looks good. But the second time I call encodeABCounts(), and the third time, and the forth, I get this...
Message Length: 28
Message: 080110001A087370656369616C41080210001A087370656369616C41
Message Length: 42
Message: 080110001A087370656369616C41080210001A087370656369616C41080310001A087370656369616C41
Message Length: 56
Message: 080110001A087370656369616C41080210001A087370656369616C41080310001A087370656369616C41080410001A087370656369616C41
...etc
So it didn't clear out the buffer/stream when the new data went in. Each time the buffer/stream is just getting longer as new data is appended.
How do I reset the stream/buffer to a state where it's ready for new data to be encoded and stuck in there, without reallocating the memory?
Thanks!
To reset the stream, simply re-create it. Now you have this:
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
You can recreate it by assigning again:
stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
Though you can also move the initial stream declaration to inside encodeABCounts() to create it every time, if you don't have any particular reason to keep it around after use. The stream creation is very lightweight, as it just stores the location and size of the buffer.
I got a client and a server application which will send each other data by using the Asio (Standalone) library. Both applications consists of two (logical) parts:
A high level part: dealing with complex objects e.g. users, permissions,...
A low level part: sending data over network between client and server
Let's assume the complex objects are already serialized by using Protocoll Buffers and the low level part of the application receives the data as std::string from the high level part. I would like to use this function from Protocoll Buffers for this job:
bool SerializeToString(string* output) const;: serializes the message
and stores the bytes in the given string. Note that the bytes are
binary, not text; we only use the string class as a convenient
container.
And say I transfer this data with async_write on the client side:
size_t dataLength = strlen(data);
//writes a certain number of bytes of data to a stream.
asio::async_write(mSocket,
asio::buffer(data, dataLength),
std::bind(&Client::writeCallback, this,
std::placeholders::_1,
std::placeholders::_2));
How can I read this data on the server side? I don't know how much data I will have to read. Therefore this will not work (length is unknown):
asio::async_read(mSocket,
asio::buffer(mResponse, length),
std::bind(&Server::readCallback, this,
std::placeholders::_1,
std::placeholders::_2));
What is the best way to solve this problem? I could think of two solutions:
Append a 'special' character at the end of data and read until I reach this 'end of data signal'. The problem is, what if this character appears in data somehow? I don't know how Protocoll Buffers serializes my data.
Send a binary string with size_of_data + data instead of data. But I don't know how to serialize the size in an platform independent way, add it to the binary data and extract it again.
Edit: Maybe I could use this:
uint64_t length = strlen(data);
uint64_t nwlength = htonl(length);
uint8_t len[8];
len[0] = nwlength >> 56;
len[1] = nwlength >> 48;
len[2] = nwlength >> 40;
len[3] = nwlength >> 32;
len[4] = nwlength >> 24;
len[5] = nwlength >> 16;
len[6] = nwlength >> 8;
len[7] = nwlength >> 0;
std::string test(len);
mRequest = data;
mRequest.insert(0, test);
and send mRequest to the server? Any traps or caveats with this code?
How could I read the length on server side and the content afterwards?
Maybe like this:
void Server::readHeader(){
asio::async_read(mSocket,
asio::buffer(header, HEADER_LENGTH),
std::bind(&Server::readHeaderCallback, this,
std::placeholders::_1,
std::placeholders::_2),
asio::transfer_exactly(HEADER_LENGTH));
}
void Server::readHeaderCallback(const asio::error_code& error,
size_t bytes_transferred){
if(!error && decodeHeader(header, mResponseLength)){
//reading header finished, now read the content
readContent();
}
else{
if(error) std::cout << "Read failed: " << error.message() << "\n";
else std::cout << "decodeHeader failed \n";
}
}
void Server::readContent(){
asio::async_read(mSocket,
asio::buffer(mResponse, mResponseLength),
std::bind(&Server::readContentCallback, this,
std::placeholders::_1,
std::placeholders::_2),
asio::transfer_exactly(mResponseLength));
}
void Server::readContentCallback(const asio::error_code& error,
size_t bytes_transferred){
if (!error){
//handle content
}
else{
//#todo remove this cout
std::cout << "Read failed: " << error.message() << "\n";
}
}
Please note that I try to use transfer_exactly. Will this work?
When sending variable length messages over a stream-based protocol, there are generally three solutions to indicate message boundaries:
Use a delimiter to specify message boundaries. The async_read_until() operations provide a convenient way to read variable length delimited messages. When using a delimiter, one needs to consider the potential of a delimiter collision, where the delimiter appears within the contents of a message, but does not indicate a boundary. There are various techniques to handle delimiter collisions, such as escape characters or escape sequences.
Use a fixed-length header with a variable-length body protocol. The header will provide meta-information about the message, such as the length of the body. The official Asio chat example demonstrates one way to handle fixed-length header and variable-length body protocols.
If binary data is being sent, then one will need to consider handling byte-ordering. The hton() and ntoh() family of functions can help with byte-ordering. For example, consider a protocol that defines the field as two bytes in network-byte-order (big-endian) and a client reads the field as a uint16_t. If the value 10 is sent, and a little-endian machine reads it without converting from network-order to local-order, then the client will read the value as 2560. The Asio chat example avoids handling endianness by encoding the body length to string instead of a binary form.
Use the connection's end-of-file to indicate the end of a message. While this makes sending and receiving messages easy, it limits the sender to only one message per connection. To send an additional message, one would need to established another connection.
A few observations about the code:
The Protocol Buffers' SerializeToString() function serializes a message to a binary form. One should avoid using text based functions, such as strlen(), on the serialized string. For instance, strlen() may incorrectly determine the length, as it will treat the first byte with a value of 0 as the terminating null byte, even if that byte is part of the encoded value.
When providing an explicitly sized buffer to an operation via asio::buffer(buffer, n), the default completion condition of transfer_all will function the same as transfer_exactly(n). As such, the duplicate use of variables can be removed:
asio::async_read(mSocket,
asio::buffer(header, HEADER_LENGTH),
std::bind(&Server::readHeaderCallback, this,
std::placeholders::_1,
std::placeholders::_2));
The htonl() overloads support uint16_t and uint_32t, not uint64_t.
Asio supports scatter/gather operations, allowing a receive operation to scatter-read into multiple buffers, and transmit operations can gather-write from multiple buffers. As such, one does not necessarily need to have both the fixed-length header and message-body contained with a single buffer.
std::string body_buffer;
body.SerializeToString(&body_buffer);
std::string header_buffer = encode_header(body_buffer.size());
// Use "gather-write" to send both the header and data in a
// single write operation.
std::vector<boost::asio::const_buffer> buffers;
buffers.push_back(boost::asio::buffer(header_buffer));
buffers.push_back(boost::asio::buffer(body_buffer));
boost::asio::write(socket_, buffers);
client must call
socket.shutdown(asio::ip::tcp::socket::shutdown_both);
socket.close();
on the server size read until EOF detected
std::string reveive_complete_message(tcp::socket& sock)
{
std::string json_msg;
asio::error_code error;
char buf[255];
while (1)
{
//read some data up to buffer size
size_t len = sock.read_some(asio::buffer(buf), error);
//store the received buffer and increment the total return message
std::string str(buf, len);
json_msg += str;
if (error == asio::error::eof)
{
//EOF received, the connection was closed by client
break;
}
else if (error)
{
throw asio::system_error(error);
}
}
return json_msg;
}
I am capturing some audio from my microphone using SFML.
The data is being stored in samples of type Int16*.
Int16* samples;
My question is. What should I do to this samples to stream it over a socket to be played in another place? I ask in relation of data type. Do I need to convert this Int16 array to another type? Or can I just send this Int16* as it is?
EDIT
void BroadcastRecorder::loadBufferFromSamples()
{
//m_samples is of type vector<Int16*>
if (!m_samples.empty()){
m_buffer.loadFromSamples(&m_samples[0], m_samples.size(), 1, getSampleRate());
m_samples.clear();
}
}
void Broadcaster::Send()
{
//load the buffer with the samples
if(!m_recorder->empty()){
m_recorder->loadBufferFromSamples();
const sf::SoundBuffer& buffer = m_recorder->getBuffer();
size_t dataLength = m_recorder->GetSamplesSize();
wxSocketClient * socket = new wxSocketClient(wxSOCKET_NOWAIT);
socket->Notify(false);
// ------------- DATA----------------------
wxString data = "";
wxString strToPrepend(_("--myboundary\r\nContent-Type: audio/wav\r\n"));
wxString strToAppend(_("\r\n--myboundary\r\n"));
// ------------- HEADER -----------------------
wxString header = "";
header.append("POST ");
header.append("/cgi-bin/operator/transmit");
header.append(" HTTP/1.0\r\n");
header.append("Content-Type: multipart/form-data; boundary=--myboundary\r\n");
header.append("Content-Length: " + wxString::Format(wxT("%i"),(dataLength + strToPrepend.Len() + strToAppend.Len()) ) + "\r\n");
header.append("Authorization: Basic keykeykeykey\r\n");
header.append("\r\n");
//-------------- CONNECTION ---------------
wxString host = _("192.168.50.11");
wxIPV4address * address = new wxIPV4address();
address->Hostname(host);
address->Service(8084);
if (socket->Connect(*address)){
//Write header
socket->Write(header.c_str(),header.Len());
//Write data
socket->Write(strToPrepend.c_str(),strToPrepend.Len());
const sf::Int16* samples = buffer.getSamples();
const char* bytesData = reinterpret_cast<const char*>(samples);
socket->Write(bytesData,dataLength);
socket->Write(strToAppend.c_str(),strToAppend.Len());
socket->Close();
}
delete socket;
delete address;
}
}
I am getting only some noises between gaps.
BTW. The audio is being sent to an IP camera p2 connector.
The data format is just the way your application treats them. After all you send raw bytes over a socket. And you can do it with anything you want
Int16 data;
const char* pBytesOfData = (const char*) &data;
int size = sizeof (Int16);
send( socket, pBytesOfdata, size, flags);
When the bytes arrive on the second end it is up to you to interpret them correctly. Probably you will want again treat them as Int16. You need to have a protocol (common way of communication) to do it right (maybe send size of the data at the begining of the transmission, etc).
You can also take a look on libraries that ease serialization: Boost.Asio and Boost.Serialization.
Firstly, You need to create and bind a socket. Then you have to send the data stored in "samples" to another peer by using socket API. For using socket API to send the data, you need to convert this data to char*. As send API of socket takes input of data you need to send as char*. For more information about sending you can go through this link. This is for windows. For Unix you can check the manpage for send API for unix.
Int16* is a pointer. The samples you get should also have an associated length. Your data will likely be between addresses: [samples, samples + length) (where samples is the address to the first sample).
To play the samples remotely (actual code will depend on what APIs you use):
open socket
in a loop
get samples from your microphone
transmit the data over socket
on the server, you will have to read samples in a loop and send them to whatever sound output API you use.
Sockets work with bytes, so in the end you will send bytes. As long as the way you interpret these bytes on the receiving side matches the data you sent, you can send whatever you want in those bytes.
In this case sending the samples directly without conversion seems the most trivial thing to do, but you will probably need to send the size of the sample before, most likely in a fixed length format, for example:
[size on 4 bytes][sample on `size` bytes]
[] [] [] [][] [] [] [] [] []
SOLVED: I was dumb. First argument of encrypt should have been key.size() and first argument of decrypt should have been RSA_size(myKey).
ORIGINAL QUESTION
Hey guys, I'm having some trouble figuring out how to do this.
Basically I just want a client and server to be able to send each other encrypted messages.
This is going to be incredibly insecure because I'm trying to figure this all out so I might as well start at the ground floor.
So far I've got all the keys working but encryption/decryption is giving me hell.
I'll start by saying I am using C++ but most of these functions require C strings so whatever I'm doing may be causing problems.
Note that on the client side I receive the following error in regards to decryption.
error:04065072:rsa routines:RSA_EAY_PRIVATE_DECRYPT:padding check failed
I don't really understand how padding works so I don't know how to fix it.
Anywho here are the relevant variables on each side followed by the code.
Client:
RSA *myKey; // Loaded with private key
// The below will hold the decrypted message
unsigned char* decrypted = (unsigned char*) malloc(RSA_size(myKey));
/* The below holds the encrypted string received over the network.
Originally held in a C-string but C strings never work for me and scare me
so I put it in a C++ string */
string encrypted;
// The reinterpret_cast line was to get rid of an error message.
// Maybe the cause of one of my problems?
if(RSA_private_decrypt(sizeof(encrypted.c_str()), reinterpret_cast<const unsigned char*>(encrypted.c_str()), decrypted, myKey, RSA_PKCS1_OAEP_PADDING)==-1)
{
cout << "Private decryption failed" << endl;
ERR_error_string(ERR_peek_last_error(), errBuf);
printf("Error: %s\n", errBuf);
free(decrypted);
exit(1);
}
Server:
RSA *pkey; // Holds the client's public key
string key; // Holds a session key I want to encrypt and send
//The below will hold the encrypted message
unsigned char *encrypted = (unsigned char*)malloc(RSA_size(pkey));
// The reinterpret_cast line was to get rid of an error message.
// Maybe the cause of one of my problems?
if(RSA_public_encrypt(sizeof(key.c_str()), reinterpret_cast<const unsigned char*>(key.c_str()), encrypted, pkey, RSA_PKCS1_OAEP_PADDING)==-1)
{
cout << "Public encryption failed" << endl;
ERR_error_string(ERR_peek_last_error(), errBuf);
printf("Error: %s\n", errBuf);
free(encrypted);
exit(1);
}
Let me once again state, in case I didn't before, that I know my code sucks but I'm just trying to establish a framework for understanding this.
I'm sorry if this offends you veteran coders.
Thanks in advance for any help you guys can provide!
Maybe not the only problem but: The first argument to RAS_xxxcrypt functions is the number of bytes of the buffers. sizeof(key.c_str()) does not yield the number of bytes in key, it yields the size of the type of key.c_str()'s result type, i.e. sizeof(const char*). You probably want to pass the number of chars in the string instead, which can be obtained with the size() member function.
I am doing some socket stuff on Symbian, which works fine so far. However,
I am facing a weird problem when trying to read out the data that has been sent.
Assume the code looks as follows:
TSockXfrLength len;
iSocket.RecvOneOrMore( buff, 0, iStatus, len );
User::WaitForRequest(iStatus);
if (iStatus == KErrNone)
{
printf(_L("Bytes received 1st try %4d..."), len);
printf(_L("Bytes Length received 2nd try %4d..."), &len);
}
Output in both cases is something with 7450 although I received exactly 145 bytes.
I can check that with a network analyser. Anyone knows what I am doing wrong here that
I do not get the proper bytes that have been received?
EDIT:
I am connecting to the socket in the following way:
TInetAddr serverAddr;
TUint iPort=445;
TRequestStatus iStatus;
TSockXfrLength len;
TInt res = iSocketSrv.Connect();
res = iSocket.Open(iSocketSrv,KAfInet,KSockStream, KProtocolInetTcp);
serverAddr.SetPort(iPort);
serverAddr.SetAddress(INET_ADDR(192,100,81,54));
iSocket.Connect(serverAddr,iStatus);
User::WaitForRequest(iStatus);
Hope that helps ;)
Thanks
Try
printf(_L("Bytes received 1st try %4d..."), len());
The TSockXfrLength type is actually a typedef of TPckgBuf<TInt>. This is the Symbian Descriptor way of storing arbitrary simple data in a 8-bit descriptor. To retrieve the value from the len object you need to use the () operator.
More information about TPckg* classes are available in the symbian developer library.