How to solve segmentation fault problems happening in support libraries? - c++

Question

I have a very odd problem going on. I can replicate the problem by the following small sample code:
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <iostream>
void printSSLErrors()
{
int l_err = ERR_get_error();
while(l_err!=0)
{
std::cout << "SSL ERROR: " << ERR_error_string(l_err, NULL) << std::endl;
l_err = ERR_get_error();
}
}
int main(int argc, char* argv[]) {
SSL_library_init();
SSL_load_error_strings();
// context
SSL_CTX* mp_ctx;
if(!(mp_ctx = SSL_CTX_new(SSLv23_server_method())))
{
printSSLErrors();
return 0;
}
std::cout << "CTX created OK" << std::endl;
// set certificate and private key
if(SSL_CTX_use_certificate_file(mp_ctx, argv[1], SSL_FILETYPE_PEM)!=1)
{
printSSLErrors();
return 0;
}
std::cout << "Certificate intialised OK" << std::endl;
if(SSL_CTX_use_PrivateKey_file(mp_ctx, argv[2], SSL_FILETYPE_PEM)!=1)
{
printSSLErrors();
return 0;
}
std::cout << "Key intialised OK" << std::endl;
SSL_CTX_free(mp_ctx);
ERR_free_strings();
}
This program works as expected when I compile it and link it using -lssl. The problem however is that the openssl routines are part of an application that also links in the mysqlclient libraries. I now recompile the above code with -lssl -lmysqlclient (note that I don't include or use anything from that library here). If I execute the program again I get a segmentation fault in the open ssl library. The most I can pull out of gdb is:
[Thread debugging using libthread_db enabled]
[New Thread -1208158528 (LWP 32359)]
CTX created OK
Certificate intialised OK
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1208158528 (LWP 32359)]
0x001b1213 in X509_get_pubkey () from /lib/libcrypto.so.4
(gdb) backtrace
#0 0x001b1213 in X509_get_pubkey () from /lib/libcrypto.so.4
#1 0x00de8a6c in SSL_rstate_string () from /lib/libssl.so.4
#2 0x086f2258 in ?? ()
#3 0xbffceb64 in ?? ()
#4 0x086f1130 in ?? ()
#5 0xbffceaa8 in ?? ()
#6 0x086f2258 in ?? ()
#7 0x086f0d90 in ?? ()
#8 0x00df4858 in ?? () from /lib/libssl.so.4
#9 0x086f2258 in ?? ()
#10 0x086f1130 in ?? ()
#11 0xbffceaa8 in ?? ()
#12 0x00de9d50 in SSL_CTX_use_PrivateKey_file () from /lib/libssl.so.4
Previous frame inner to this frame (corrupt stack?)
(gdb) frame 0
#0 0x001b1213 in X509_get_pubkey () from /lib/libcrypto.so.4
For some reason this only happens when I use mysqlclient v 15 and not with mysqlclient v 16. This is probably too obscure for anyone to solve, but some comments on how linking against a dynamic library that the code itself doesn't even use can cause these errors would be very helpful.
The system is:
RHEL ES4, gcc 3.4.6, openssl-0.9.7a, MySQL-5.11
Any thoughts?
Edit: Here is the output to possibly clarify things a little more:
[Lieuwe ~]$ c++ openssl_test.cpp -lssl -o ssltest
[Lieuwe ~]$ ./ssltest /etc/httpd/conf/certs/test.crt /etc/httpd/conf/certs/test.key
CTX created OK
Certificate intialised OK
Key intialised OK
[Lieuwe ~]$ c++ openssl_test.cpp -lmysqlclient -lssl -o ssltest
[Lieuwe ~]$ ./ssltest /etc/httpd/conf/certs/test.crt /etc/httpd/conf/certs/test.key
CTX created OK
Certificate intialised OK
Segmentation fault (core dumped)
[Lieuwe ~]$
Note that for this purpose I use the crt and key file that the apache server also uses (and work)
Edit 2: Here is the (relevant?) output of valgrind for the program
CTX created OK
--5429-- REDIR: 0x5F6C80 (memchr) redirected to 0x4006184 (memchr)
Certificate intialised OK
==5429== Invalid read of size 4
==5429== at 0xCF4205: X509_get_pubkey (in /lib/libcrypto.so.0.9.7a)
==5429== by 0xDE8A6B: (within /lib/libssl.so.0.9.7a)
==5429== by 0xDE9D4F: SSL_CTX_use_PrivateKey_file (in /lib/libssl.so.0.9.7a)
==5429== by 0x8048C77: main (in /home/liwu/ssltest)
==5429== Address 0x4219940 is 0 bytes inside a block of size 84 free'd
==5429== at 0x4004EFA: free (vg_replace_malloc.c:235)
==5429== by 0xC7FD00: CRYPTO_free (in /lib/libcrypto.so.0.9.7a)
==5429== by 0xCE53A7: (within /lib/libcrypto.so.0.9.7a)
==5429== by 0xCE5562: ASN1_item_free (in /lib/libcrypto.so.0.9.7a)
==5429== by 0xCE0560: X509_free (in /lib/libcrypto.so.0.9.7a)
==5429== by 0xDE979E: SSL_CTX_use_certificate_file (in /lib/libssl.so.0.9.7a)
==5429== by 0x8048C23: main (in /home/liwu/ssltest)
==5429==
==5429== Invalid read of size 4
==5429== at 0xCD4A5F: EVP_PKEY_copy_parameters (in /lib/libcrypto.so.0.9.7a)
==5429== by 0xDE8A7C: (within /lib/libssl.so.0.9.7a)
==5429== by 0xDE9D4F: SSL_CTX_use_PrivateKey_file (in /lib/libssl.so.0.9.7a)
==5429== by 0x8048C77: main (in /home/liwu/ssltest)
==5429== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==5429==
==5429== Process terminating with default action of signal 11 (SIGSEGV)
==5429== Access not within mapped region at address 0x0
==5429== at 0xCD4A5F: EVP_PKEY_copy_parameters (in /lib/libcrypto.so.0.9.7a)
==5429== by 0xDE8A7C: (within /lib/libssl.so.0.9.7a)
==5429== by 0xDE9D4F: SSL_CTX_use_PrivateKey_file (in /lib/libssl.so.0.9.7a)
==5429== by 0x8048C77: main (in /home/liwu/ssltest)
==5429==

Answer 1

I would suggest running your program under Valgrind. Valgrind is intended to provide help with exactly this kind of problem and it is generally much easier to use than a debugger.
If I were to hazard a guess, I would first suspect a memory error in your application (or, less likely, in one of the shared libraries) that is sensitive to the memory layout of the resulting executable. Adding a new shared library or, say, enabling debugging options could very well make the problem appear or disappear for no apparent reason.

Answer 2

The only logical explanation may be that the public key, which is needed for X509_get_pubkey(), can not be located.
Can you please verify that the public key requested by the function is available?

Answer 3

I'd think that the mysql client library is linked against another version of libssl. If you are on linux: are both libraries installed via your distro's official repositories? Are you linking against the static (.a) or dynamic (.so) versions of those libraries?
You can play around with the nm command to find out more (read the manpage).
You can try to rebuild the mysql client library yourself to make sure the same libssl version is used and see whether the problem disappears.