I'm using this decryption function to get the plain text value of a cipher which was encrypted using EVP AES 265 GCM; I can see data in rawOut but ret = EVP_DecryptFinal_ex(ctx, rawOut, &len); returns 0; can you provide any insight as to why? I've also seen sources which do rawOut + len in the EVP_DecryptFinal_ex code, I'm not sure why this would be needed as it would move the pointer to the end of the buffer.
unsigned char* keyDecrypter(unsigned char* pszMasterKey)
{
ERR_load_crypto_strings();
int ret, len;
EVP_CIPHER_CTX* ctx;
unsigned char* rawOut = new unsigned char[48]; // ToDo Remove Hardcoded Value
Info info = m_header.processKeyInfo();
if (NULL == info.nonce)
return NULL;
if (!(ctx = EVP_CIPHER_CTX_new()))
return NULL;
if (!EVP_DecryptInit_ex(ctx, EVP_aes_256_gcm(), NULL, pszMasterKey, info.nonce))
return NULL;
if (!EVP_DecryptUpdate(ctx, NULL, &len, m_header.aad, m_header.aad_len))
return NULL;
if (!EVP_DecryptUpdate(ctx, rawOut, &len, m_header.encryptedValue, m_header.encryptedValueLen))
return NULL;
// Finalise the decryption. A positive return value indicates success,
// anything else is a failure - the plain text is not trustworthy.
ret = EVP_DecryptFinal_ex(ctx, rawOut, &len);
ERR_print_errors_fp(stderr);
EVP_CIPHER_CTX_free(ctx);
if (ret > 0)
{
return rawOut;
}
else
{
return NULL;
}
}
From the OpenSSL doc:
"EVP_DecryptFinal() will return an error code if padding is enabled and the final block is not correctly formatted."
Apparently, the padding scheme between encryption and decryption do not match, or perhaps the size of ciphertext fed into the decryption engine did not exactly match the size of the ciphertext that was output from the encryption engine. Note that the ciphertext must include the result of a corresponding call to EVP_EncryptFinal_ex.
It is unfortunate that the original poster did not provide sufficient information to make an exact determination.
You need to pass rawOut + len to EVP_DecryptFinal_ex. See in the example at the end of the documentation:
/* Buffer passed to EVP_EncryptFinal() must be after data just
* encrypted to avoid overwriting it.
*/
if(!EVP_EncryptFinal_ex(ctx, outbuf + outlen, &tmplen))
{
/* Error */
return 0;
}
outlen += tmplen;
Also note that rawOut must have enough room for (m_header.aad_len + cipher_block_size) bytes. You can get the block size with EVP_CIPHER_block_size().
I have a function for receiving messages of variable length through TCP. The send-function creates a buffer, puts the length of message in first four bytes, fills the rest with the message, and sends by parts. But the receive-function was receiving 4 bytes less. And suddenly, when I put one printf, everything is working as it should.
bool TCP_Server::recvMsg(SOCKET client_sock, std::unique_ptr<char[]>& buf_ptr, int* buf_len)
{
int msg_len;
int rcvd = 0, tmp;////
/* get msg len */
if((tmp = recv(client_sock, (char*)&msg_len, sizeof(msg_len), 0)) == -1)
{
handle_error("recv");
return false;
}
*buf_len = msg_len;
printf("msg_len = %d\n", msg_len); //
printf("tmp getting msg_len = %d\n", tmp);//
rcvd += tmp;//
buf_ptr.reset((char*)malloc(msg_len));
if(buf_ptr.get() == nullptr) // not enough memory
{
handle_error("malloc");
return false;
}
/* get msg of specified len */
/* get by biggest available pieces */
int i = 1;
while(int(msg_len - 1440 * i) > 0)
{
char* cur_ptr = buf_ptr.get() + 1440 * (i - 1);
if((tmp=recv(client_sock, cur_ptr, 1440, 0)) == -1)
{
handle_error("recv");
return false;
}
printf("1440 = %d\n", tmp); // doesn't work if I comment this line
rcvd += tmp;
i++;
}
int rest = msg_len - 1440 * (i - 1);
/* get the rest */
if((tmp = recv(client_sock, buf_ptr.get() + msg_len - rest, rest, 0)) == -1)
{
handle_error("(recv)reading with msg_len");
return false;
}
rcvd += tmp;//
printf("rcvd = %d\n", rcvd);//
return true;
}
In sum, if I comment printf("1440 = %d\n", tmp);, the function is receiving 4 bytes less.
I'm compiling with x86 Debug.
Here's the dissimilar lines in asm(/FA flag): http://text-share.com/view/50743a5e
But I don't see anything suspicious
printf writes to the console, which is a fairly slow operation, relatively speaking. The extra delay it produces might easily change how much data has arrived in the buffer when you call recv.
As Tulon comments, reads from TCP streams can be any length. TCP doesn't preserve message boundaries, so they don't necessarily match the send sizes on the other end. And if less data has been sent across the network than you asked to read, you'll get what is available.
Solution: stop thinking of 1440 byte chunks. Get rid of i and simply compare rcvd to msg_len.
I am trying to read complete messages from my GPS via serial port.
The message I am looking for starts with:
0xB5 0x62 0x02 0x13
So I read from the serial port like so
while (running !=0)
{
int n = read (fd, input_buffer, sizeof input_buffer);
for (int i=0; i<BUFFER_SIZE; i++)
{
if (input_buffer[i]==0xB5 && input_buffer[i+1]== 0x62 && input_buffer[i+2]== 0x02 && input_buffer[i+3]== 0x13 && i<(BUFFER_SIZE-1) )
{
// process the message.
}
}
The problem I am having is that I need to get a complete message. Half of a message could be in the buffer one iteration. And the other half could come into the message the next iteration.
Somebody suggested that free the buffer up from the complete message. And then I move the rest of data in the buffer to the beginning of the buffer.
How do I do that or any other way that make sure I get every complete selected message that comes in?
edit//
I want a particular class and ID. But I can also read in the length
To minimize the overhead of making many read() syscalls of small byte counts, use an intermediate buffer in your code.
The read()s should be in blocking mode to avoid a return code of zero bytes.
#define BLEN 1024
unsigned char rbuf[BLEN];
unsigned char *rp = &rbuf[BLEN];
int bufcnt = 0;
static unsigned char getbyte(void)
{
if ((rp - rbuf) >= bufcnt) {
/* buffer needs refill */
bufcnt = read(fd, rbuf, BLEN);
if (bufcnt <= 0) {
/* report error, then abort */
}
rp = rbuf;
}
return *rp++;
}
For proper termios initialization code for the serial terminal, see this answer. You should increase the VMIN parameter to something closer to the BLEN value.
Now you can conveniently access the received data a byte at a time with minimal performance penalty.
#define MLEN 1024 /* choose appropriate value for message protocol */
unsigned char mesg[MLEN];
while (1) {
while (getbyte() != 0xB5)
/* hunt for 1st sync */ ;
retry_sync:
if ((sync = getbyte()) != 0x62) {
if (sync == 0xB5)
goto retry_sync;
else
continue; /* restart sync hunt */
}
class = getbyte();
id = getbyte();
length = getbyte();
length += getbyte() << 8;
if (length > MLEN) {
/* report error, then restart sync hunt */
continue;
}
for (i = 0; i < length; i++) {
mesg[i] = getbyte();
/* accumulate checksum */
}
chka = getbyte();
chkb = getbyte();
if ( /* valid checksum */ )
break; /* verified message */
/* report error, and restart sync hunt */
}
/* process the message */
switch (class) {
case 0x02:
if (id == 0x13) {
...
...
You can break the read into three parts. Find the start of a message. Then get the LENGTH. Then read the rest of the message.
// Should probably clear these in case data left over from a previous read
input_buffer[0] = input_buffer[1] = 0;
// First make sure first char is 0xB5
do {
n = read(fd, input_buffer, 1);
} while (0xB5 != input_buffer[0]);
// Check for 2nd sync char
n = read(fd, &input_buffer[1], 1);
if (input_buffer[1] != 0x62) {
// Error
return;
}
// Read up to LENGTH
n = read(fd, &input_buffer[2], 4);
// Parse length
//int length = *((int *)&input_buffer[4]);
// Since I don't know what size an int is on your system, this way is better
int length = input_buffer[4] | (input_buffer[5] << 8);
// Read rest of message
n = read(fd, &input_buffer[6], length);
// input_buffer should now have a complete message
You should add error checking...
I'm trying to generate a RSA Signature with libopenssl for c++:
But when I run my code, I get a SIGABRT. I did some deep debugging into libopenssl internal stuff to see where the Segfault comes from. I'll come to this later on.
First I want to make clear, that the RSA PrivateKey was successfully loaded from a .pem file. So Im pretty sure that's not the problem's origin.
So my question is: How to avoid the SIGABRT and what is the cause of it ?
I'm doing this for my B.Sc. Thesis so I really appreciate your help :)
Signature Generation Function:
DocumentSignature* RSASignatureGenerator::generateSignature(ContentHash* ch, CryptographicKey* pK) throw(PDVSException) {
OpenSSL_add_all_algorithms();
OpenSSL_add_all_ciphers();
OpenSSL_add_all_digests();
if(pK == nullptr)
throw MissingPrivateKeyException();
if(pK->getKeyType() != CryptographicKey::KeyType::RSA_PRIVATE || !dynamic_cast<RSAPrivateKey*>(pK))
throw KeyTypeMissmatchException(pK->getPem()->getPath().string(), "Generate RSA Signature");
//get msg to encrypt
const char* msg = ch->getStringHash().c_str();
//get openssl rsa key
RSA* rsaPK = dynamic_cast<RSAPrivateKey*>(pK)->createOpenSSLRSAKeyObject();
//create openssl signing context
EVP_MD_CTX* rsaSignCtx = EVP_MD_CTX_create();
EVP_PKEY* priKey = EVP_PKEY_new();
EVP_PKEY_assign_RSA(priKey, rsaPK);
//init ctxt
if (EVP_SignInit(rsaSignCtx, EVP_sha256()) <=0)
throw RSASignatureGenerationException();
//add data to sign
if (EVP_SignUpdate(rsaSignCtx, msg, std::strlen(msg)) <= 0) {
throw RSASignatureGenerationException();
}
//create result byte signature struct
DocumentSignature::ByteSignature* byteSig = new DocumentSignature::ByteSignature();
//set size to max possible
byteSig->size = EVP_MAX_MD_SIZE;
//alloc buffer memory
byteSig->data = (unsigned char*)malloc(byteSig->size);
//do signing
if (EVP_SignFinal(rsaSignCtx, byteSig->data, (unsigned int*) &byteSig->size, priKey) <= 0)
throw RSASignatureGenerationException();
DocumentSignature* res = new DocumentSignature(ch);
res->setByteSignature(byteSig);
EVP_MD_CTX_destroy(rsaSignCtx);
//TODO open SSL Memory leaks -> where to free open ssl stuff?!
return res;
}
RSA* rsaPK = dynamic_cast(pK)->createOpenSSLRSAKeyObject();
virtual RSA* createOpenSSLRSAKeyObject() throw (PDVSException) override {
RSA* rsa = NULL;
const char* c_string = _pem->getContent().c_str();
BIO * keybio = BIO_new_mem_buf((void*)c_string, -1);
if (keybio==NULL)
throw OpenSSLRSAPrivateKeyObjectCreationException(_pem->getPath());
rsa = PEM_read_bio_RSAPrivateKey(keybio, &rsa, NULL, NULL);
if(rsa == nullptr)
throw OpenSSLRSAPrivateKeyObjectCreationException(_pem->getPath());
//BIO_free(keybio);
return rsa;
}
SigAbrt origin in file openssl/crypto/mem.c
void CRYPTO_free(void *str, const char *file, int line)
{
if (free_impl != NULL && free_impl != &CRYPTO_free) {
free_impl(str, file, line);
return;
}
#ifndef OPENSSL_NO_CRYPTO_MDEBUG
if (call_malloc_debug) {
CRYPTO_mem_debug_free(str, 0, file, line);
free(str);
CRYPTO_mem_debug_free(str, 1, file, line);
} else {
free(str);
}
#else
free(str); // <<<<<<< HERE
#endif
}
the stacktrace
stacktrace screenshot from debugger (clion - gdb based)
I just found the Bug (and Im really not sure if this is a libopenssl bug..)
//set size to max possible
byteSig->size = EVP_MAX_MD_SIZE;
//alloc buffer memory
byteSig->data = (unsigned char*)malloc(byteSig->size);
The problem was when I set the buffer size to EVP_MAX_MD_SIZE!
The (in my opinion) very very strange thing is, that you have to keep the size uninitialized! (not even set to 0 - just "size_t size;" ).
Strange thing here is that then you also HAVE TO allocate memory just like I did. I dont understand this because then an undefined size of memory gets allocated..
What the really weird is that libopenssl internally sets the size back to 0 and allocates the memory itself.. (I detected this by browsing the libopenssl source code)
I am trying to write an application which amongst other things uses the openssl blowfish implementation (blowfish.h) to transport files over a simple server/client pair.
However, whilst some files are encrypted, transported, received and decrypted correctly, some end up being corrupted, after the final decryption stage. This leads me to think that the encryption routines are not being called correctly (since I have also tried with equivalent DES library calls, with the same 'intermittent corruption' results).
The relevant code is pasted below.
Basically, it starts with the function send_file (called by a connected client). This splits the file into chunks. Each 1024 byte chunk is encrypted separately and then sent. Each chunk is then received by the server in the receive_file function, decrypted and saved to disc.
Any idea what the problem could be? (Note if necessary, I will add the code for the whole application).
Cheers,
Ben.
void encryptHelper(const char*,int);
void decryptHelper(const char*,int);
inline void blowfish(unsigned char *data, int data_len, char* key, int enc)
{
// hash the key first!
unsigned char obuf[20];
bzero(obuf,20);
SHA1((const unsigned char*)key, strlen(key), obuf);
BF_KEY bfkey;
int keySize = strlen(key);
BF_set_key(&bfkey, 16, (const unsigned char*)obuf);
unsigned char ivec[8];
memset(ivec, 0, 8);
unsigned char out[1024];// = (unsigned char*) malloc(1024);
bzero(out,1024);
int num = 0;
BF_cfb64_encrypt(data, out, data_len, &bfkey, ivec, &num, enc);
data=out;
//memcpy(data, out, data_len);
//free(out);
}
void MyFrame::encryptHelper(char* orig, int inlength)
{
char *pb=(char*)(std::string((passInput->GetValue()).mb_str()).c_str());
blowfish((unsigned char*)orig, inlength, pb, DES_ENCRYPT);
}
void MyFrame::decryptHelper(char* orig, int inlength)
{
char *pb=(char*)(std::string((passInput->GetValue()).mb_str()).c_str());
blowfish((unsigned char*)orig, inlength, pb, DES_DECRYPT);
}
int MyFrame::send_file(int fd)
{
char rec[10];
struct stat stat_buf;
fstat (fd, &stat_buf);
int size=stat_buf.st_size;
int remSize=size;
int value=0;
while(size > 0)
{
char buffer[1030];
bzero(buffer,1030);
bzero(rec,10);
int n;
if(size>=1024)
{
value+=1024;
n=read(fd, buffer, 1024);
// encrypt is necessary
if(encButtonOn->GetValue()) encryptHelper(buffer,1024);
// Send a chunk of data
n=send(sockFile_, buffer, 1024, 0 );
// Wait for an acknowledgement
n = recv(sockFile_, rec, 10, 0 );
}
else // reamining file bytes
{
value+=size;
n=read(fd, buffer, size);
if(encButtonOn->GetValue()) encryptHelper(buffer,size);
buffer[size]='\0';
n=send(sockFile_,buffer, size, 0 );
n=recv(sockFile_, rec, 10, 0 );
}
MyFooEvent event( 0, 992 );
double firstBit = (double)value/remSize;
firstBit=firstBit*100.0;
event.adouble=firstBit;
wxPostEvent (this, event);
size -= 1024;
}
// Send a completion string
int n = send(sockFile_, "COMP",strlen("COMP"), 0 );
char buf[10];
bzero(buf,10);
// Receive an acknowledgemnt
n = recv(sockFile_, buf, 10, 0 );
return(0);
}
int MyFrame::receive_file()
{
// receive file size and send ack
char sizeBuffer[50];
bzero(sizeBuffer,50);
int n;
//read(This->sockpw,buffer,bufferSize);
n=read(sockFile_, sizeBuffer, 50);
n=send(sockFile_,"OK", strlen("OK"), 0 );
int size = atoi(sizeBuffer);
//std::cout<<size<<std::endl;
// receive file name and send ack
char saveName[256];
bzero(saveName,256);
n=read(sockFile_, saveName, 256);
n=send(sockFile_,"OK",strlen("OK"), 0 );
//std::cout<<saveName_<<std::endl;
// start file writing process to local disk
// decrypt first if necessary
std::cout<<arraySize(saveName)<<std::endl;
std::cout<<strlen(saveName)<<std::endl;
if(encButtonOn->GetValue()) decryptHelper(saveName,strlen(saveName));
ofstream outFile(saveName,ios::out|ios::binary|ios::app);
// vars for status gauge
int remSize=size;
int value=0;
while(size > 0)
{
// buffer for storing incoming data
char buf[1030];
bzero(buf,1030);
if(size>=1024)
{
value+=1024; // for status gauge
// receive chunk of data
n=recv(sockFile_, buf, 1024, 0 );
// decrypt if necessary
if(encButtonOn->GetValue()) decryptHelper(buf,1024);
// write chunk of data to disk
outFile.write(buf,1024);
// send acknowledgement
n = send(sockFile_, "OK", strlen("OK"), 0 );
}
else
{
value+=size;
n=recv(sockFile_, buf, size, 0 );
if(encButtonOn->GetValue()) decryptHelper(buf,size);
buf[size]='\0';
outFile.write(buf,size);
n = send(sockFile_, "OK", strlen("OK"), 0 );
}
// Update status gauge
MyFooEvent event( 0, 992 );
double firstBit = (double)value/remSize;
firstBit=firstBit*100.0;
event.adouble=firstBit;
wxPostEvent (this, event);
size -= 1024;
}
outFile.close();
// Receive 'COMP' and send acknowledgement
// ---------------------------------------
char buf[10];
bzero(buf,10);
n = recv(sockFile_, buf, 10, 0 );
n = send(sockFile_, "OK", strlen("OK"), 0 );
std::cout<<"File received..."<<std::endl;
// Display image event
MyFooEvent eventF( 0, 995 );
eventF.SetText(wxString(saveName, wxConvUTF8));
wxPostEvent (this, eventF);
return(0);
}
I'm assuming that:
char *pb=(char*)(std::string((passInput->GetValue()).mb_str()).c_str());
blowfish((unsigned char*)orig, inlength, pb, DES_DECRYPT);
decrypts into pb, which is actually the buffer of a temporary string. You simply cannot use std::string like this. The fact that you had to use so many casrs to do this shouldhave been a warning - good C and C++ code does not normally require casts at all. Basically, you need to rethink what you are doing.
not sure, could be a buffer overrun somewhere or memory corruption...
you could use valgrind to detect the issue or perhaps try simplifying the conversions/...
Having fixed a few bugs by asking a few other questions, I have gotten the file encryption process working, but only when the client and server are both on the same localhost machine. When they reside on different machines, the file still ends up being corrupted. I think it is due to the fact that send_file and receive file are called from threads as follows:
void
*MyFrame::send_fileT(void* tid)
{
accessHelper* ah = static_cast<accessHelper*>(tid);
MyFrame* This = ah->This;
This->send_file(fileSendID);
pthread_exit(NULL);
}
void
*MyFrame::receive_fileT(void* tid)
{
accessHelper* ah = static_cast<accessHelper*>(tid);
MyFrame* This = ah->This;
This->receive_file();
pthread_exit(NULL);
}
....and then the receive_file or send_file functions are calling the blowfish function to carry out the encryption. Now if a function is called within a pthread (i.e. send_file and receive_file), then if that function calls another function (i.e. encryptHelper -- blowfish), is it possible that the calling function will not 'properly' wait for the called function to finish correctly?
Fixed:
n=read(fd, buffer, 2048);
if(enc)encryptHelper(buffer,n);
n=send(sockFile_, buffer, n, 0 );
[called in a loop]
The problem was, was that it cannot be ensured that all n bytes of the encrypted buffer are transferred. Thus only some of the encrypted bytes are sent leading to inconsistent decryption on the receiving end.