We are programmatically creating PDF using our in house lib (C++) by adding all the required objects so that PDF readers can render them properly.
Currently we are enhancing the lib to support digital signatures in PDF. Our users will use USB token or Windows certificates to sign the PDF.
On studying raw PDF file with digital signature, we were able to make sense of all the objects except for the contents of Sig type object.
18 0 obj
<<
/Type /Sig
/Filter /Adobe.PPKLite --> signature handler for authenticating the fields contents
/SubFilter /adbe.pkcs7.sha1 --> submethod of the handler
/Contents <....> --> signature token
/ByteRange [ 0 101241 105931 7981
] --> byte range for digest calc
/M (D:20210505094125+05'30') --> timestamp
/Reason ()
/Location ()
/ContactInfo ()
>>
endobj
We have referred
https://www.adobe.com/devnet-docs/acrobatetk/tools/DigSigDC/Acrobat_DigitalSignatures_in_PDF.pdf
to understand what all constitutes the signature token.
We need direction on how to programmatically create the signature token for PDF using windows APIs. Currently we are not looking at 3rd party lib solutions.
Thanks in advance.
Update
We tried the following:
Updated our in-house PDF lib to support incremental updates so that digital signing related objects can be added. We added something like this apart from the obj# 18 mentioned above:
16 0 obj --> new Acroform obj
<<
/Fields [ 17 0 R ]
/SigFlags 3
>>
endobj
2 0 obj --> Updating root to add AcroForm
<<
/Type /Catalog
/Pages 3 0 R
/AcroForm 16 0 R
>>
endobj
17 0 obj --> new obj for signature field
<<
/T (SignatureField1)
/Type /Annot
/Subtype /Widget
/FT /Sig
/F 4
/Rect [ 270 159 503 201 ] --> field position. this will have image of sign
/P 5 0 R
/V 18 0 R
/AP <<
/N 19 0 R
>>
>>
endobj
5 0 obj --> updating existing page obj with Annots
<<
/Type /Page
/Parent 3 0 R
/MediaBox [ 0 0 595 841 ]
/Resources 4 0 R
/Contents 6 0 R
/Annots [ 17 0 R ]
>>
endobj
18 0 obj
<<
/Type /Sig
/Filter /Adobe.PPKLite
/SubFilter /adbe.pkcs7.sha1 --> we tried with adbe.pkcs7.detached as well
/Contents <> --> updated contents using windows APIs
/ByteRange [ 0 100381 102645 7322
] --> updated ByteRange with right offsets and lengths
/M (D:20210610125837+05'30') --> sign verified time
/Reason ()
/Location ()
/ContactInfo ()
>>
endobj
19 0 obj --> new obj
<<
/Length 7
/BBox [ 0 0 233 42 ]
/Type /XObject
/Subtype /Form
/Resources <<
/XObject <<
/FRM 20 0 R
>>
>>
>>
stream
/FRM Do
endstream
endobj
20 0 obj --> new obj for image manipulation
<<
/Length 29
/Type /XObject
/Subtype /Form
/Resources <<
/XObject <<
/Im1 21 0 R
>>
>>
/BBox [ 0 0 233 42 ]
>>
stream
q 233 0 0 42 0 0 cm /Im1 Do Q
endstream
endobj
21 0 obj --> image obj which contains sign info. Generated by us
<<
/Length 6166
/Type /XObject
/Subtype /Image
/Width 372
/Height 82
/ColorSpace /DeviceRGB
/BitsPerComponent 8
/Filter /DCTDecode
>>
stream
---------------------------------> image stream
endstream
endobj
xref --> updated xref
0 1
0000000000 65535 f
2 1
0000099954 00000 n
5 1
0000100172 00000 n
16 6
0000099901 00000 n
0000100020 00000 n
0000100297 00000 n
0000102944 00000 n
0000103096 00000 n
0000103271 00000 n
trailer --> updated trailer
<<
/Root 2 0 R
/Info 1 0 R
/Size 22
/ID [ <982AAACB948CE1AD9FDD976D177BF316> <982AAACB948CE1AD9FDD976D177BF316> ]
--> ID generated via windows API
/Prev 99491
>>
startxref
109605
%%EOF
For contents data, we used the below API:
bool SignMessageBySubjectName (BytePtr pMessage, ULong pMessageSize, StrPtr pSubjectName, CRYPT_DATA_BLOB * pSignBlob)
{
HCERTSTORE store_handle = NULL;
PCCERT_CONTEXT cert_context = NULL;
BYTE * signed_blob = NULL;
ULong signed_blob_size;
ULong message_size;
CRYPT_SIGN_MESSAGE_PARA signature_params;
BYTE * message;
pSignBlob->cbData = 0;
pSignBlob->pbData = NULL;
message = (BYTE *) pMessage;
message_size = (pMessageSize + 1) * sizeof(Char); //Size in bytes
const BYTE * message_array[] = {message};
DWORD message_array_size[1];
message_array_size[0] = message_size;
store_handle = CertOpenStore(CERT_STORE_PROV_SYSTEM, 0, NULL,
CERT_SYSTEM_STORE_CURRENT_USER, L"MY");
cert_context = CertFindCertificateInStore( store_handle, PKCS_7_ASN_ENCODING | X509_ASN_ENCODING, 0,
CERT_FIND_SUBJECT_STR, pSubjectName, NULL);
signature_params.cbSize = sizeof(CRYPT_SIGN_MESSAGE_PARA);
signature_params.dwMsgEncodingType = PKCS_7_ASN_ENCODING | X509_ASN_ENCODING;
signature_params.pSigningCert = cert_context;
signature_params.HashAlgorithm.pszObjId = szOID_RSA_SHA1RSA;
signature_params.HashAlgorithm.Parameters.cbData = NULL;
signature_params.cMsgCert = 1;
signature_params.rgpMsgCert = &cert_context;
signature_params.cAuthAttr = 0;
signature_params.dwInnerContentType = 0;
signature_params.cMsgCrl = 0;
signature_params.cUnauthAttr = 0;
signature_params.dwFlags = 0;
signature_params.pvHashAuxInfo = NULL;
signature_params.rgAuthAttr = NULL;
//Get size of signed message
CryptSignMessage(&signature_params, TRUE, 1, message_array, message_array_size,NULL, &signed_blob_size);
signed_blob = (BYTE *) Malloc(signed_blob_size);
CryptSignMessage(&signature_params, TRUE, 1, message_array, message_array_size, signed_blob, &signed_blob_size);
pSignBlob->cbData = signed_blob_size;
pSignBlob->pbData = signed_blob;
CertFreeCertificateContext(cert_context);
CertCloseStore(store_handle, CERT_CLOSE_STORE_FORCE_FLAG);
return true;
}
While using CryptSignMessage() with detached parameter as TRUE, we get a around 850 length sign token which we convert to hex and add in the contents part. It'll approximately be around 1700 chars.
In case of the image used in the Field newly added, we generated our own image and added it as a PDF obj.
For the ID in trailer part, we generated the same using API from Bcrypt.lib (BCryptGenRandom()), converted its output to hex and updated the ID part.
Listing out the steps we did:
We generated 2 buffers. Both buffers are identical with respect to all the PDF objects required, the ID generated from BCryptGenRandom() and ByteRange array updated with actual values. buffer1 has contents data as 0s for a definite length acting as a placeholder. buffer2 has empty contents data (/Contents <>)
buffer2 will be passed onto CryptSignMessage() to generate the sign token. This will be converted to hex.
The hex sign token will be added to contents part of buffer1 replacing the 0s based on its length.
buffer1 will be written to a PDF file.
When we did all these, and opened the PDF in readers, we got errors like
Signature is invalid
Document has been corrupted or altered since the signature was applied.
Error from a PDF Reader:
Detailed Error:
But with these errors too, the reader was able to identify the user, certificate, hash algorithm and signature algorithm used.
We think we need to somehow add timestamp data as part of the sign token so as to avoid this error. Or something else we would have missed.
PFA sample PDF here:https://drive.google.com/file/d/1Udog4AmGoq2ls3Tu3Wq5s2xU9LxaI3fH/view?usp=sharing
Kindly help us solve this issue. Thanks in advance.
We used a different set of APIs to make this work.
Pasting the code here:
bool SignatureHandler::SignMessageTest (BytePtr pMessage, ULong pMessageSize, StrPtr pSubjectName, CRYPT_DATA_BLOB * pSignBlob, LPSTR pOid, DWORD pFlag, DWORD pType)
{
HCERTSTORE store_handle = NULL;
PCCERT_CONTEXT cert_context = NULL;
BYTE * signed_blob = NULL;
ULong signed_blob_size = 0;
CRYPT_SIGN_MESSAGE_PARA signature_params;
BYTE * message;
BOOL rc;
pSignBlob->cbData = 0;
pSignBlob->pbData = NULL;
store_handle = CertOpenStore (CERT_STORE_PROV_SYSTEM, 0, NULL, CERT_SYSTEM_STORE_CURRENT_USER, L"MY");
cert_context = CertFindCertificateInStore (store_handle, (PKCS_7_ASN_ENCODING | X509_ASN_ENCODING), 0, CERT_FIND_SUBJECT_STR, pSubjectName, NULL);
HCRYPTPROV_OR_NCRYPT_KEY_HANDLE a = 0;
DWORD ks = 0;
BOOL bfr = false;
HCRYPTPROV_OR_NCRYPT_KEY_HANDLE PrivateKeys;
CERT_BLOB CertsIncluded;
CMSG_SIGNER_ENCODE_INFO Signers;
HCRYPTMSG hMsg;
rc = CryptAcquireCertificatePrivateKey (cert_context, 0, 0, &a, &ks, &bfr);
CMSG_SIGNER_ENCODE_INFO SignerEncodeInfo = {0};
SignerEncodeInfo.cbSize = sizeof (CMSG_SIGNER_ENCODE_INFO);
if (a)
SignerEncodeInfo.hCryptProv = a;
if (bfr)
PrivateKeys = a;
CERT_BLOB SignerCertBlob;
SignerCertBlob.cbData = cert_context->cbCertEncoded;
SignerCertBlob.pbData = cert_context->pbCertEncoded;
CertsIncluded = SignerCertBlob;
SignerEncodeInfo.cbSize = sizeof (CMSG_SIGNER_ENCODE_INFO);
SignerEncodeInfo.pCertInfo = cert_context->pCertInfo;
SignerEncodeInfo.dwKeySpec = ks;
SignerEncodeInfo.HashAlgorithm.pszObjId = pOid;
SignerEncodeInfo.HashAlgorithm.Parameters.cbData = NULL;
SignerEncodeInfo.pvHashAuxInfo = NULL;
Signers = SignerEncodeInfo;
CMSG_SIGNED_ENCODE_INFO SignedMsgEncodeInfo = {0};
SignedMsgEncodeInfo.cbSize = sizeof (CMSG_SIGNED_ENCODE_INFO);
SignedMsgEncodeInfo.cSigners = 1;
SignedMsgEncodeInfo.rgSigners = &Signers;
SignedMsgEncodeInfo.cCertEncoded = 1;
SignedMsgEncodeInfo.rgCertEncoded = &CertsIncluded;
SignedMsgEncodeInfo.rgCrlEncoded = NULL;
signed_blob_size = 0;
signed_blob_size = CryptMsgCalculateEncodedLength ((PKCS_7_ASN_ENCODING | X509_ASN_ENCODING), pFlag, pType, &SignedMsgEncodeInfo, 0, pMessageSize);
if (signed_blob_size) {
signed_blob_size *= 2;
hMsg = CryptMsgOpenToEncode (CERTIFICATE_ENCODING_TYPE,
pFlag,
pType,
&SignedMsgEncodeInfo,
0,
NULL);
if (hMsg) {
signed_blob = (BYTE *)malloc (signed_blob_size);
BOOL CU = CryptMsgUpdate (hMsg, (BYTE *)pMessage, (DWORD)pMessageSize, true);
if (CU) {
if (CryptMsgGetParam (
hMsg, // Handle to the message
CMSG_CONTENT_PARAM, // Parameter type
0, // Index
signed_blob, // Pointer to the BLOB
&signed_blob_size)) // Size of the BLOB
{
signed_blob = (BYTE *)realloc (signed_blob, signed_blob_size);
if (hMsg) {
CryptMsgClose (hMsg);
hMsg = 0;
}
}
}
if (hMsg)
CryptMsgClose (hMsg);
hMsg = 0;
}
}
CryptReleaseContext (a, 0);
pSignBlob->cbData = signed_blob_size;
pSignBlob->pbData = signed_blob;
CertFreeCertificateContext (cert_context);
CertCloseStore (store_handle, CERT_CLOSE_STORE_FORCE_FLAG);
return true;
}
The oid, flag and type we used are szOID_RSA_SHA1RSA, CMSG_DETACHED_FLAG and CMSG_SIGNED respectively.
On converting pSignBlob->pbData to hex and adding it to /Contents, the PDF and signature became valid when opened in PDF readers.
Ok, the signature container is embedded correctly.
But there are issues with the signature container itself:
Both in the SignedData.digestAlgorithms collection and in the SignerInfo.digestAlgorithm value you have used the OID of SHA1withRSA, but that is a full signature algorithm, not the mere digest algorithm SHA1 expected there.
Then the SHA1 hash of the signed bytes is BB78A402F7A537A34D6892B83881266501A691A8 but the hash you signed is 90E28B8A0D8E48691DAFE2BA10A4761FFFDCCD3D. This might be because you hash buffer2 and
buffer2 has empty contents data (/Contents <>)
The hex string delimiters '<' and '>' also belong to the contents value and, therefore, must also be removed in buffer2.
Furthermore, your signature is very weak:
It uses SHA1 as hash algorithm. SHA1 meanwhile has been recognized as too weak a hash algorithm for document signatures.
It doesn't use signed attributes, neither the ESS signing certificate nor the algorithm identifier protection attribute. Many validation policies require such special attributes.
I'd like to make a program in python to slice the string with length 23. I have a code that solves the purpose. I would like to know if there is a better and simple way to do this?
prtNum = "88-90-25-65-13-20-20-41-88-90-25-65-13-20-20-41-88-91-31-81-81-20-20-44-88-91-31-81-82-20-20-44-88-91-31-82-81-20-20-44-88-91-31-81-83-20-20-44"
len = len(prtNum)
length = 23
start = 0
for i in range(0, len):
sw = prtNum[start:length]
len1 = length + 1
start = len1
length = length+ 24
print(sw)
output looks like this:
88-90-25-65-13-20-20-41
88-90-25-65-13-20-20-41
88-91-31-81-81-20-20-44
88-91-31-81-82-20-20-44
88-91-31-82-81-20-20-44
88-91-31-81-83-20-20-44
I used rrdtool python extension for save data in rrd:
## Creating db.
rrdtool.create(rrd_file,
'--step', '2',
'DS:%s:GAUGE:4:U:U' % DSNAME,
'RRA:AVERAGE:0,5:1:288',
)
value = 23
for i in range(4):
rrdtool.update('/home/way/workspace/RrdDaemon/test', "%s:%s" % (datetime_2_sec(str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))), str(value)))
sleep(2)
Cycle worked 4 times, and i want to get 4 points. But i get 3 only:
1460382646: -nan
1460382648: 2,3000000000e+01
1460382650: 2,3000000000e+01
1460382652: 2,3000000000e+01
1460382654: -nan
I tried to change heartbeat, step , xff - nothing helps me. Now i try with 1 iteration:
for i in range(1):
rrdtool.update('/home/way/workspace/RrdDaemon/test', "%s:%s" % (datetime_2_sec(str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))), str(value)))
Timestamp : 1460385371
Result:
1460385368: -nan
1460385370: -nan
1460385372: -nan
sudo rrdtool info test:
filename = "test"
rrd_version = "0003"
step = 2
last_update = 1460385371
header_size = 584
ds[ds0].index = 0
ds[ds0].type = "GAUGE"
ds[ds0].minimal_heartbeat = 3
ds[ds0].min = NaN
ds[ds0].max = NaN
ds[ds0].last_ds = "23"
ds[ds0].value = NaN
ds[ds0].unknown_sec = 1
rra[0].cf = "AVERAGE"
rra[0].rows = 288
rra[0].cur_row = 65
rra[0].pdp_per_row = 1
rra[0].xff = 0,0000000000e+00
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
Do i make anything wrong or its the way which rrd working for?
Thank you.
The problem you are running into is that you are crossing from an unknown state into a known state. The minimal_heartbeat defines the maximum interval permissible between two updates, for rrdtool to consider the time in between the updates to contain valid data.
This also means that the first update after a period of unknown data only serves to indicate the time when the data becomes known again ... the next update (within the interval defined by the minimal_heartbeat).
I try to use prepared select for get data from mysql,beacuse I think this faster than regular select.
this is select syntax:
char *sql = "select id,d1,d2,d3,d4,d5 from pricelist where d1 > ? limit 1000000";
that id,d2,d3 type unsigned int and other __int64
I wirte my code for prepared like below:
stmt = mysql_stmt_init(conn);
mysql_stmt_prepare(stmt, sql, strlen(sql));
// Select
param[0].buffer_type = MYSQL_TYPE_LONG;
param[0].buffer = (void *) &myId;
param[0].is_unsigned = 1;
param[0].is_null = 0;
param[0].length = 0;
// Result
result[0].buffer_type = MYSQL_TYPE_LONG;
result[0].buffer = (void *) &id;
result[0].is_unsigned = 1;
result[0].is_null = &is_null[0];
result[0].length = 0;
result[1].buffer_type = MYSQL_TYPE_LONGLONG;
result[1].buffer = (void *) &d1;
result[1].is_unsigned = 1;
result[1].is_null = &is_null[0];
result[1].length = 0;
result[2].buffer_type = MYSQL_TYPE_LONG;
result[2].buffer = (void *) &d2;
result[2].is_unsigned = 1;
result[2].is_null = &is_null[0];
result[2].length = 0;
result[3].buffer_type = MYSQL_TYPE_LONG;
result[3].buffer = (void *) &d3;
result[3].is_unsigned = 1;
result[3].is_null = &is_null[0];
result[3].length = 0;
result[4].buffer_type = MYSQL_TYPE_LONGLONG;
result[4].buffer = (void *) &d4;
result[4].is_unsigned = 1;
result[4].is_null = &is_null[0];
result[4].length = 0;
result[5].buffer_type = MYSQL_TYPE_LONGLONG;
result[5].buffer = (void *) &d5;
result[5].is_unsigned = 1;
result[5].is_null = &is_null[0];
result[5].length = 0;
mysql_stmt_bind_param(stmt, param);
mysql_stmt_bind_result(stmt, result);
mysql_stmt_execute(stmt);
mysql_stmt_store_result(stmt);
while(mysql_stmt_fetch (stmt) == 0){
}
and my code for reqular select is like below:
mysql_query(conn,"select id ,d1,d2,d3,d4,d5 from pricebook where us > 12 limit 1000000")
result = mysql_use_result(conn);
while (mysql_fetch_row(result)){
}
I run this two functions from remote pc and check time period for each one,duration for both of then is same equal to 6 sec
and when I check pcap file I see that vol that sent for prepared is same with reqular query even that in prepared comperes data.
$ capinfos prepared.pcap regular.pcap
File name: prepared.pcap
File type: Wireshark - pcapng
File encapsulation: Ethernet
Packet size limit: file hdr: (not set)
Number of packets: 40 k
File size: 53 MB
Data size: 52 MB
Capture duration: 6 seconds
Start time: Thu Aug 22 09:41:54 2013
End time: Thu Aug 22 09:42:00 2013
Data byte rate: 8820 kBps
Data bit rate: 70 Mbps
Average packet size: 1278.63 bytes
Average packet rate: 6898 packets/sec
SHA1: 959e589b090e3354d275f122a6fe6fbcac2351df
RIPEMD160: 7db6a437535d78023579cf3426c4d88d8ff3ddc3
MD5: 888729dc4c09baf736df22ef34bffeda
Strict time order: True
File name: regular.pcap
File type: Wireshark - pcapng
File encapsulation: Ethernet
Packet size limit: file hdr: (not set)
Number of packets: 38 k
File size: 50 MB
Data size: 49 MB
Capture duration: 6 seconds
Start time: Thu Aug 22 09:41:05 2013
End time: Thu Aug 22 09:41:11 2013
Data byte rate: 7740 kBps
Data bit rate: 61 Mbps
Average packet size: 1268.65 bytes
Average packet rate: 6101 packets/sec
SHA1: badf2040d826e6b0cca089211ee559a7c8a29181
RIPEMD160: 68f3bb5d4fcfd640f2da9764ff8e9891745d4800
MD5: 4ab73a02889472dfe04ed7901976a48c
Strict time order: True
if this ok that duration is same or I don't use prepared select as well as?
how I can improve it?
thanks.
The database server executes prepared statements and regular statements with the same speed. The performance difference comes when you execute the same query with different parameters: a prepared statement is parsed and prepared for execution once and then can be executed cheaply with different parameters, while a regular statement has to be parsed every time you want to execute it.
I wrote the following code to get the physical media serial number but in one of my computers it returns null instead.
Does anybody know what the problem is?
Thanks.
var searcher = new ManagementObjectSearcher("SELECT * FROM Win32_PhysicalMedia");
foreach( ManagementObject mo in searcher.Get() )
{
Console.WriteLine("Serial: {0}", mo["SerialNumber"]);
}
The Serial Number is optional, defined by the manufacturer, and for your device it is either blank or unsupported by the driver.
Virtually all hard drives have a serial number, but most USB-style Flash memory sticks do not (generally a cost issue). I would imagine most unbranded CD/DVD/BD discs would also be non-serialized.
Here is the code I used, the serial number somehow is returned raw with each pair of chars reversed (strange) and using Win32_PhysicalMedia gave different results if I ran the code as a user or an Administrator(more strange) - Windows 7 Ultimate, VS 2008 using VB only:
Function GetHDSerial() As String
Dim strHDSerial As String = String.Empty
Dim index As Integer = 0
Dim Data As String
Dim Data2 As String
Dim ndx As Integer
Dim query As New SelectQuery("Win32_DiskDrive")
Dim search As New ManagementObjectSearcher(query)
Dim info As ManagementObject
Try
For Each info In search.Get()
Data = info("SerialNumber")
Data2 = ""
For ndx = 1 To Data.Length - 1 Step 2
Data2 = Data2 & Chr(Val("&H" & Mid(Data, ndx, 2)))
Next ndx
Data = String.Empty
For ndx = 1 To Data2.Length - 1 Step 2
Data = Data & Mid(Data2, ndx + 1, 1) & Mid(Data2, ndx, 1)
Next
Data2 = Data
If Len(Data) < 8 Then Data2 = "00000000" 'some drives have no s/n
Data2 = Replace(Data2, " ", "") ' some drives pad spaces in the s/n
'forget removeable drives
If InStr(info("MediaType").ToString, "Fixed", CompareMethod.Text) > 0 Then
strHDSerial = strHDSerial & "Drive " & index.ToString & " SN: " & Data2 & vbCrLf
index += 1
End If
Next
Catch ex As Exception
strHDSerial = "Error retrieving SN for Drive "
msgbox(index.ToString)
End Try
Return strHDSerial
End Function