Differences in reading private key with libp11 and pkcs11-engine - c++

I'm trying to implement SSL client authentication in C++ using credentials stored on a smartcard.
In essence, this means using the openssl library to initialize an SSL context using a certificate and a private key, then using this context for all future https connections.
The libraries i've come across to help me with this are libp11 and its pkcs11-engine module, found here: https://github.com/OpenSC/libp11 .
A scenario in which my code actually works as intended with our webserver is where the certificate is listed and retrieved via libp11 and the private key is retrieved by id using the pkcs11 engine:
PKCS11_enumerate_certs(slot->token, &certs, &ncerts);
X509 *cert = certs[0].x509;
EVP_PKEY *pkey = ENGINE_load_private_key(pkcs11_eng, "pkey_id", NULL, NULL);
if (!SSL_CTX_use_certificate(context->ctx, cert )) {
throw SSLError::getLastError();
}
if (!SSL_CTX_use_PrivateKey(context->ctx, pkey )) {
throw SSLError::getLastError();
}
if (!SSL_CTX_check_private_key(context->ctx)) {
throw SSLError::getLastError();
}
However, for consistency and to keep things simple, it is preferred to use libp11 for both of these retrievals as this would also eliminate the use of a whole other component (the pkcs11-engine).
The issue i'm having is that when retrieving pkey with:
PKCS11_KEY *key = PKCS11_find_key(&certs[0]);
EVP_PKEY pkey = PKCS11_get_private_key(key)
And initializing ssl, the checks pass, but the following error is thrown by the SSL_connect() function:
error:80009005:Vendor defined:PKCS11_rsa_encrypt:General Error
Basically the private key works when it's retrieved via engine but an error is thrown when using libp11, which is strange because looking through the code on github for the engine, i noticed the same p11 calls being made that i was using.
If anyone has any experience with this topic and might know what's going on here, it would help me enormously.

This was my bad. I was initializing the pkcs11 engine with our own dll implementing pkcs11 but using the opensc dll when initializing p11. I guess the lesson learned here is always look carefully before copy/pasting code

Related

Boost asio ssl: password callback not called if private key passed with context::use_private_key

I'm writing a test unit that uses the boost asio ssl.
I'm using Boost 1.54 on Ubuntu 14.04 64 bit.
I plan to make the test self-sufficient and not rely on files to specify the private key, so I want to hard encode the key and its password in the test itself (they are just test key and password).
The code is below. For now it does nothing but I'm just trying to make the password callback work when the private key is specified:
std::string password_callback(
std::size_t max_length,
boost::asio::ssl::context::password_purpose purpose)
{
return "test";
}
TEST(StreamReader, sslStream)
{
std::string certificate = "-----BEGIN CERTIFICATE-----\n\
MIIFJjCCAw4CCQDQjrFrRcdRkjANBgkqhkiG9w0BAQsFADBVMQswCQYDVQQGEwJT\n\
BLABLABLABLA";
std::string key = "-----BEGIN RSA PRIVATE KEY-----\n\
Proc-Type: 4,ENCRYPTED\n\
DEK-Info: DES-EDE3-CBC,06622C22CAB27AC2\n\
\n\
JMudxXy4ZxB733xh7QO4elsVCTzJZuWl9Go4ZMuWx0DZb2fYHqXynKZSf7UactSw\n\
vhKJnLPZaa5U+xOr9cdpSd3SwtQyNu6yaVQH3af2ILRwUsw9mQmI8yqIIF1Y6AgV\n\
BLABLABLABLA";
boost::asio::io_service io_service;
boost::asio::ssl::context ctx(boost::asio::ssl::context::tlsv12);
ctx.set_password_callback(password_callback);
ctx.use_certificate(boost::asio::const_buffer(certificate.c_str(), certificate.size()), boost::asio::ssl::context::pem);
ctx.use_private_key(boost::asio::const_buffer(key.c_str(), key.size()), boost::asio::ssl::context::pem);
ctx.set_verify_mode(boost::asio::ssl::verify_peer);
}
When use_private_key is executed then the password callback is not called and I have to enter the password manually in the console. If I replace use_private_key with use_private_key_file then the callback is called.
I would expect password_callback to be called also when use_private_key is used.
Am I missing something?
This reflects a limitation of the underlying OpenSSL API:
Encyption applies to loading keys from file only:
The private keys loaded from file can be encrypted. In order to successfully load encrypted keys, a function returning the passphrase must have been supplied, see ssl_ctx_set_default_passwd_cb(3). (Certificate files might be encrypted as well from the technical point of view, it however does not make sense as the data in the certificate is considered public anyway.)
(from ssl_ctx_use_privatekey_file)
Consider using the raw key from inside the application. The raw key should be difficult to spot from looking at a binary. Of course you can scatter the key around in bits and encrypt them manually, but keep in mind a determined attacker can always figure it out anyways.
The rationale behind this could be that passphrase protected key files can be passed around safely, as long as the passphrase is not passed at the same time. As soon as you embed the key in the program that also contains the passphrase used to decrypt it, passing it around is inherently unsafe anyways.
UPDATE
In fact it should not be difficult to get the desired behaviour by patching Boost Asio: The use_private_key context member function ends up calling ::PEM_read_bio_PrivateKey which does support a password callback.
You could add it:
//evp_private_key.p = ::PEM_read_bio_PrivateKey(bio.p, 0, 0, 0);
evp_private_key.p = ::PEM_read_bio_PrivateKey(bio.p, 0,
handle_->default_passwd_callback,
handle_->default_passwd_callback_userdata); // SEHE WAS HERE
to get the behaviour you want.
Be sure to ping the Asio developer/boost mailing lists whether they're interested in adding this as a feature.
I have to update this thread because I was in the same situation. It seems that support for this is now implemented:
https://github.com/boostorg/asio/blob/a2204b67f85cf63936a1daaacee830b6b69cec9f/include/boost/asio/ssl/impl/context.ipp#L845
So good news!

How to consume SOAP Webservices in Blackberry

I am working on a blackberry application. I need to call soap webservices, but I am unable to do so, and am getting null as a response. Following is my code:
private static final String CONNECTION_PARAMS = ";deviceside=true";
SoapObject request = new SoapObject("http://service.action.com/",
"findActiveSecurities");
SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);
envelope.bodyOut = request;
HttpTransportBasicAuth ht =
new HttpTransportBasicAuth("http://myurl.com/ebclient/services/MobileClientService?wsdl"+CONNECTION_PARAMS,
"myusername",
"mypassword");
PropertyInfo propInfo=new PropertyInfo();
propInfo.type=PropertyInfo.INTEGER_CLASS;
//adding parameters
request.addProperty("arg0","NSE");
request.addProperty("arg1","0");
request.addProperty("arg2","100");
envelope.setOutputSoapObject(request);
try {
ht.call(SOAP_ACTION, envelope);
result = (SoapObject)envelope.getResponse();
System.out.println(result);
} catch (IOException e) {
e.printStackTrace();
} catch (XmlPullParserException e) {
e.printStackTrace();
}
I am getting null as a result.Please have a look at the code and help me to correct it.
Thanks in advance
Actually the problem was instead of passing 0 and 100 as String ...
request.addProperty("arg0","NSE");
request.addProperty("arg1","0");
request.addProperty("arg2","100");
I use
request.addProperty("arg0","NSE");
request.addProperty("arg1",new Integer(0));
request.addProperty("arg2",new Integer(1000));
also this link helped me.
also before asking this question I was facing some problem that the Simulator was not recognizing a Library. It shows error message something like "there is no library Ksoap2_j2me.jar" - resolved from this link.
Sorry for poor English but I think this can save time of some other developer.
It's hard to tell from what you're posted, but my guess is that you're having some kind of network problem. I'm guessing that you initialize result = null;, and then your call to ht.call() throws an IOException, leaving result null.
You're using ksoap2, which is a library written for generic J2ME clients. However, BlackBerry networking doesn't work exactly like all other J2ME platforms.
You are controlling the BlackBerry network transport with your connection params string, which is hardcoded:
private static final String CONNECTION_PARAMS = ";deviceside=true";
Unfortunately, this string suffix may not be right for all network conditions (or any, if you don't have device APN settings correct).
I think you have a couple choices:
1. Connection Suffix Strings
You can try dynamically choosing the right suffix string, depending on conditions when your app runs. This can allow the device, for example, to connect via Wi-Fi if it's available, or via BES if that's available. Developers new to BlackBerry may be surprised that app code needs to worry about this (read here for more, or watch this).
If you want to simply replace CONNECTION_PARAMS with a dynamic string, you might check out the implementation here.
2. ConnectionFactory
In OS 5.0, BlackBerry added the ConnectionFactory class, which was a big improvement over the old way of having to assemble connection strings. If you only need to support OS 5.0 and greater, you might choose to rewrite the code to use ConnectionFactory.
Since you have access to the ksoap source code, you could change it. It looks like the connection code is in ServiceConnectionMidp.java:
public ServiceConnectionMidp(String url) throws IOException {
connection = (HttpConnection) Connector.open(url, Connector.READ_WRITE, true);
}
Instead of attaching connection parameters to the url passed to this class, you could change the class to get the connection from a ConnectionFactory, customized to support the network transports you want.
Doing this means that if you ever want to update your code to use a new version of ksoap2, you'll need to make these modifications again. However, given the future of BlackBerry Java, that seems like a reasonable compromise to make.

Silent accept .pfx certificates

Such a problem: I need to import .pfx certificates in my installation (WiX 3.5). I need to accept importing them without any(!) user interaction, even acception (need to be so for testion on building servers and testing makets)
Tried standard WiX solution, using WiXIISExtension, but there is no option for quiet(silent) import.
Tried such solutions on C++ (deferred custom actions from dll in wix):
Chain PfxImportCertStore - CertEnumCertificatesStore - CertAddCertificateContextToStore:
Certificate manager asked for acception...
CryptUIWizImport with flag CRYPTUI_WIZ_NO_UI: CryptoAPI tried to access to a private key, and manager asked for acception...
Is there any way of solving this? or I really need to write a custom service for acception?
Well, I solved my problem. In my case it was just the flag of user protected certificate. Just my dummy. but, in these two days I realised:
Be carefully with the methods of importing certificate. You should know different situations, when you need to use import from file or from context.
Pay attention to the flags, you used (the more if you use some examples)
Be sure your certificate has approvals.
P.S. Comment me if somebody needs good examples of working codes of importing different certificates.
I got the same problem when trying to add a self signed certificate in Root trust location. It was prompting me the security warning.
Security Warning
With the following code I am now able to add the self signed certificate silently in the Root trust location.
HCERTSTORE hUIRootCertStore = CertOpenStore(CERT_STORE_PROV_SYSTEM, 0, NULL, CERT_SYSTEM_STORE_LOCAL_MACHINE, L"Root");
CRYPTUI_WIZ_IMPORT_SRC_INFO importSrc;
memset(&importSrc, 0, sizeof(CRYPTUI_WIZ_IMPORT_SRC_INFO));
importSrc.dwSize = sizeof(CRYPTUI_WIZ_IMPORT_SRC_INFO);
importSrc.dwSubjectChoice = CRYPTUI_WIZ_IMPORT_SUBJECT_FILE;
importSrc.pwszFileName = L"<certificate file>";
importSrc.pwszPassword = L"<your password>";
importSrc.dwFlags = CRYPT_EXPORTABLE;
bRet = CryptUIWizImport(CRYPTUI_WIZ_NO_UI | CRYPTUI_WIZ_IMPORT_TO_LOCALMACHINE, NULL, NULL, &importSrc, hUIRootCertStore);
If there is any other method please tell me.

Getting certificate chain to a private root

I'm trying to verify that the certificate from a signature chains back to a particular root certificate, which is not trusted by Windows (it's a private certificate for the app).
My current attempt to do this involves creating a chaining engine which only trusts the specific certificate I want as the root, so that no other chains can be generated.
HCERTSTORE hPrivateRootStore = CertOpenStore(CERT_STORE_PROV_FILENAME, dwEncoding,
NULL, CERT_STORE_OPEN_EXISTING_FLAG | CERT_STORE_READONLY_FLAG,
_T("C:\\Test\\PrivateRoot.cer"));
CERT_CHAIN_ENGINE_CONFIG config;
memset(&config, 0, sizeof(config));
config.cbSize = sizeof(config);
config.hRestrictedTrust = hPrivateRootStore;
config.dwFlags = CERT_CHAIN_CACHE_ONLY_URL_RETRIEVAL | CERT_CHAIN_ENABLE_SHARE_STORE;
HCERTCHAINENGINE hEngine;
CertCreateCertificateChainEngine(&config, &hEngine);
CERT_CHAIN_PARA params;
memset(&params, 0, sizeof(params));
params.cbSize = sizeof(params);
PCCERT_CHAIN_CONTEXT chains = NULL;
if (CertGetCertificateChain(hEngine, pCertContext, NULL, hStore, &params,
0, NULL, &chains))
...
(error checking omitted for clarity; pCertContext and hStore came from CryptQueryObject extracting the signature and related certificates from a signed binary file.)
Unfortunately, this doesn't seem to work; despite using a custom chaining engine it still seems to be searching the OS store, and either doesn't find a chain or finds one to a different root (which is trusted by the OS). I can only get the chain I want by adding my private root certificate to the OS trusted store.
I've also tried setting config.hRestrictedOther to an empty memory store, since the docs suggest that having hRestrictedTrust non-NULL will bring in the system stores again, but that doesn't make any difference.
Is there something I'm missing, or a better way to do this?
Edit: just to give a bit more context, I'm trying to do something similar to the driver signing certificates, where the signing certificate chains back to two different roots: one standard CA root trusted by the OS and one internal root (which in drivers is also trusted by the OS but in my case will only be trusted by my app). The cross occurs somewhere mid-way up the "main" chain; there could potentially be a number of different files all signed with different "real" CAs but still chained back to my internal certificate.
I've found a half-baked workaround now; it's a little ugly but it does kinda work. I got the basic idea from Chromium's test suite; it involves installing a hook into Crypt32 such that when it tries to open the system stores to build the chain it gets my custom store instead, containing only my desired certificate.
The good is that this seems to force CertGetCertificateChain to go "past" the real CA certificate and chain all the way to my custom certificate, instead of stopping at the CA certificate (which is what it usually does when that is trusted).
The bad is that it doesn't seem to stop it from building chains to and trusting any other CA certificate. I can work around that by explicitly verifying that the root of the chain is the certificate I wanted, but it's less than ideal, and I'm not sure if there are situations which will trip it up.
Still looking for any better solutions; I'm definitely getting the impression that I'm taking the wrong path somewhere.
Ok, new plan. I'm now just walking the chain manually (since I know that all of the certificates I care about will be in the hStore extracted from the signed .exe), with basic structure like this:
Use WinVerifyTrust to do basic "is it not tampered" authentication.
Use CryptQueryObject to obtain certificate store hStore from the .exe
Use CryptMsgGetParam and CertFindCertificateInStore to find the signing certificate from hStore.
Starting from the signing certificate, use CertFindCertificateInStore with CERT_FIND_SUBJECT_NAME in a loop to find the potential issuer certificates; keep walking back until I hit a self-signed certificate or find my desired root (checking for a match via CertComparePublicKeyInfo).
Abandon a particular branch if CertVerifySubjectCertificateContext says that the signatures don't match.
Seems cleaner than the previous approaches I was trying. Thoughts/comments/alternatives?
(Something which in some ways does seem to make more sense would be to add an additional custom countersignature [similar to the timestamping] instead of trying to chain certificates like this, but I can't find any information on doing that, or what the pros/cons are.)

What's the correct way to verify an SSL certificate in Win32?

I want to verify an SSL certificate in Win32 using C++. I think I want to use the Cert* API so that I can get the benefit of the Windows certificate store. This is what I've come up with.
Is it correct?
Is there a better way to do this?
Am I doing anything wrong?
bool IsValidSSLCertificate( PCCERT_CONTEXT certificate, LPWSTR serverName )
{
LPTSTR usages[] = { szOID_PKIX_KP_SERVER_AUTH };
CERT_CHAIN_PARA params = { sizeof( params ) };
params.RequestedUsage.dwType = USAGE_MATCH_TYPE_AND;
params.RequestedUsage.Usage.cUsageIdentifier = _countof( usages );
params.RequestedUsage.Usage.rgpszUsageIdentifier = usages;
PCCERT_CHAIN_CONTEXT chainContext = 0;
if ( !CertGetCertificateChain( NULL,
certificate,
NULL,
NULL,
&params,
CERT_CHAIN_REVOCATION_CHECK_CHAIN,
NULL,
&chainContext ) )
{
return false;
}
SSL_EXTRA_CERT_CHAIN_POLICY_PARA sslPolicy = { sizeof( sslPolicy ) };
sslPolicy.dwAuthType = AUTHTYPE_SERVER;
sslPolicy.pwszServerName = serverName;
CERT_CHAIN_POLICY_PARA policy = { sizeof( policy ) };
policy.pvExtraPolicyPara = &sslPolicy;
CERT_CHAIN_POLICY_STATUS status = { sizeof( status ) };
BOOL verified = CertVerifyCertificateChainPolicy( CERT_CHAIN_POLICY_SSL,
chainContext,
&policy,
&status );
CertFreeCertificateChain( chainContext );
return verified && status.dwError == 0;
}
You should be aware of RFC3280 section 6.1 and RFC5280 section 6.1. Both describe algorithms for validating certificate paths. Even though Win32 API takes care of some things for you, it could still be valuable to know about the process in general.
Also, here’s a (in my opinion) pretty trustworthy reference: Chromium certificate verification code.
Overall, I think your code isn't incorrect. But here’s a few things I’d look into/change, if I were you:
1. Separate Common Name Validation
Chromium validates certificate common name separately from the chain. Apparently they've noticed some problems with it. See the comments for their rationale:
cert_verify_proc.win.cc:731 // Certificate name validation happens separately, later, using an internal
cert_verify_proc.win.cc:732 // routine that has better support for RFC 6125 name matching.
2. Use CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT
Chromium also uses the CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT flag instead of CERT_CHAIN_REVOCATION_CHECK_CHAIN. I actually started to looking into this before I found their code, and it reinforced my belief that you should use CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT.
Even though both aforementioned RFCs specify that a self-signed trust anchor is not considered part of a chain, the documentation for CertGetCertificateChain (http://msdn.microsoft.com/en-us/library/windows/desktop/aa376078(v=vs.85).aspx) says it builds a chain up to, if possible, a trusted root certificate. A trusted root certificate is defined (on the same page) as a trusted self-signed certificate.
This eliminates the possibility that *EXCLUDE_ROOT might skip revocation checking for a non-root trust anchor (Win32 actually requires trust-anchors to be self-signed, even though it is not required by any RFCs. Though this is not officially documented).
Now, since a root CA certificate can not revoke itself (the CRL could not be signed/verified), it seems to me that these two flags are identical.
I did some googling and stumbled across this forum post: http://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/9f95882a-1a68-477a-80ee-0a7e3c7ae5cf/x509revocationflag-question?forum=windowssecurity. A member of .NET Product Group (supposedly) claims that the flags in practice act the same, if the root is self-signed (in theory, the ENTIRE_CHAIN flag would check the root certificate for revocation if it included a CDP extension, but that can’t happen).
He also recommends to use the *EXCLUDE_ROOT flag, because the other flag could cause an unnecessary network request, if the self-signed root CA includes the CDP extension.
Unfortunately:
I can’t find any officially documented explanation on the differences between the two flags.
Even though it is likely that the linked discussion applies to the same Win32 API flags under the hood of .NET, it is not guaranteed.
To be completely sure that it’s ok to use CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT, I googled a bit more and found the Chromium SSL certificate verification code I linked to at the top of my reply.
As an added bonus, the Chromium cert_verify_proc_win.cc file contains the following hints about IE verification code:
618: // IE passes a non-NULL pTime argument that specifies the current system
619: // time. IE passes CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT as the
620: // chain_flags argument.
Not sure how they’d know this, but at this point I’d feel comfortable using CERT_CHAIN_REVOCATION_CHECK_EXCLUDE_ROOT.
3. Different Accepted Certificate Usages
I noticed Chromium also specifies 3 certificate usages instead of 1:
szOID_PKIX_KP_SERVER_AUTH,
szOID_SERVER_GATED_CRYPTO,
szOID_SGC_NETSCAPE
From what I can gather through Google, the other usages can be required by older web browsers, otherwise they can fail to establish a secure connection.
If Chromium deems fit to include these usages, I'd follow suit.
Note that if you change your code, you should also set params.RequestedUsage.dwType to USAGE_MATCH_TYPE_OR instead of USAGE_MATCH_TYPE_AND.
—
I can’t think of any other comments at the moment. But if I were you, I’d check out Chromium source myself (and maybe Firefox too) - just to be sure I haven’t missed anything.
I think the best answer depends on what exactly you are attempting to do.
I will caution you that SSL is based on the assumption that Both endpoints want a secure connection. If either endpoint isn't interested in maintaining security then there is none.
Its a trivial effort to put byte codes in your distributed code that simply returns true for this function. That's why windows moved a lot of validation into the kernel. But they didn't anticipate people running windows on virtual hardware, which makes circumventing the OS just about as trivial.
Now consider that you expect to be provided a cert from some source, but pretending that that source couldn't be provided the same information from a reliable source. And then hand it to you. So You cannot rely on certificates to "prove" anyone is anyone in particular.
The only protection gained from certificates are in preventing outsiders, not endpoints, from breaching the confidentiality of the message being transported.
Any other use is doomed to fail, and it will fail eventually with potentially catastrophic results.
Sorry for the big post. The comment section has a word limit.
The functions CertGetCertificateChain and CertVerifyCertificatePolicy go together. This part is correct.
For CertGetCertificateChain the flag can be set to any of the following three if you want to check for revocation:
CERT_CHAIN_REVOCATION_CHECK_END_CERT
CERT_CHAIN_REVOCATION_CHECK_CHAIN
CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT.
Only one of them can be used, these three options cannot be ORed. Beside one of these flags you can consider how the chain should be created; using local cache or just CRL or OCSP. For these considerations read this link.
Error in executing the function or more simply if the return value is 0, it does not mean the certificate is invalid, rather you were unable to perform the operation. For error information use GetLastError(). So your logic of returning false is wrong, it is more of a case of throwing the error and let the client code decide whether to try again or go on to do other stuff.
In this link there is a section called "classify the error", please read that. Basically you should check certChainContext->TrustStatus.dwErrorStatus. Here a list of error statuses will be ORed. Please check CERT_TRUST_STATUS msdn reference. So here you can have your business logic. For example, if you find the error status of the value (CERT_TRUST_REVOCATION_STATUS_UNKNOWN | CERT_TRUST_IS_OFFLINE_REVOCATION) that certificate revocation check could not be performed, you have the option to decide what you want (let the cert go or still mark it as invalid).
So, before going to call CertVerifyCertificatePolicy you have the option to discard or already flag a validation error.
If you choose to come to CertVerifyCertificatePolicy, the chromium code is a wonderful reference regarding how to map policy_status.dwError to your error class/enum.