I'm developing an open source project and I have been working on making the builds reproducible so that my users can compare the checksums of the binaries that I distribute with their own builds (if they were to build the project with/from the source code).
Unfortunately, new versions of Windows and MacOS use code signing in order to check binaries and prevent their execution if they aren't signed (I'm aware that there are ways to override this and execute the binary anyways, but this is not user friendly).
I'd like to sign the binaries that I distribute so that my users can run them without any problems. But I'm not sure if that is possible to do while also keeping the reproducible builds.
For a build to be reproducible, the end user must have all the tools / source code required to build the project and, once compiled, the end result should be the same bit-a-bit binary compared to the one that I'm distributing. But that would mean that I'd have to distribute the private key / cert used to sign the binary, which is not a good idea for multiple reasons.
Is there a way to have both reproducible builds and signed binaries?
Here is a general approach for creating reproducible signed builds for open source.
Create your "sign binaries" script as follows:
Compile project
Make a checksum of the project's signing preimage
Compare the checksum to the file SIGNING_CHECKSUM
If there is a match:
Use the signature in SIGNING_SIGNATURE to build the package
Verify that SIGNING_SIGNATURE was signed by SIGNING_PUBLIC_KEY
If there is not a match:
Save the checksum to the file SIGNING_CHECKSUM
Use the local signing key to sign the binary
Save the signature to the file SIGNING_SIGNATURE
Verify that SIGNING_SIGNATURE was signed by SIGNING_PUBLIC_KEY
End result is that anybody can reproducibly build the untampered source code. Anybody can edit the source code and reproducibly build the binary so long as the modifications to not modify the signing preimage. And only the developer with access to the signing key is able to sign new releases.
Related
I am maintaining a scientific library as a C++ repository on GitHub, using Cmake for builds.
Now the problem is the following:
There is a lot of code in the repository that the end users do not want to download. This includes e.g. support code to produce precomputed coefficients and calculation parameters. The end users only want to download the minimal amount of files that are necessary for them to do their job. The motivation for this is to minimize the size of the library and to make it easier for the end users to understand the code they want to use.
My question is:
Is it possible to create a "Release Version" of a library in a GitHub repository, and if yes, how should this be done properly?
If you add .gitattributes to your repository, you can add files/wildcards in similar fashion as in .gitignore but then you can add export-ignore after the file entry. Files that then match these ignored patterns wont be added to the release source archive.
I think that it would be a good idea to have a track of the build file of a project. But, I've read that build files should not be public, since they can contain "sensible" information.
Is recommendable to have the build file in the same repository of the source code? Even if the repo is public? If it is not recommendable, would I have to have an independent repo for each project only for the build file?
Edit: the projects would be plugins for Moodle. And, I've never used build systems before, as I guess you have already deduced.
You didn't mention what kind of build system you're using. Nevertheless, having the build file along the project source has advantages:
The build file is versioned the same way the source code is
You don't have to keep track of which build file belongs to which project
You shouldn't have any sensible information within a build file - most systems allow you do have encrypted authentication information in a build file. In the worst case, you'll have to keep the information stored elsewhere.
I'm wondering if there's a macro or a simple way to let the compiler increment either major, minor or revision of my code each time when I compile?
By the way I'm using the ARM compiler and uVision from Keil.
To set the version is not a compiler topic. This should be done in connection with a source code control / version control system like cvs/svn/git or others. Your build id should be connected to the content of your source code database to get reproducible builds from checkouts from your version control system. Or, if your code is not already committed to your database a dirty-tag should be provided and compiled in to give the user of the software a chance to see that this is not a controlled version.
Simply counting a value in a variable can be done by a Makefile or in pre- and post-build instructions which depends on the used IDE. Sorry, for keil I have no experience...
Define Post-Build Event to run small external program. The program has to modify specific .h file. In the header file define macros like VER_MAJOR, VER_MINOR, VER_BUILD. Date/time string can also be updated. I use this method and can control the version numbers as I wish.
IMO, you do not need to do this, especially increase a number every time you compile your code.
Set the major and minor revision manually in a header file; you should not have to do this often.
Build number should only be related to the source control revision number (i.e. you should be able to build and rebuild any revision under source control).
Imagine if you are a team of 5 developers, and everyone build and rebuild on their side what is the actual build number?
do they all update a header file ?
who is responsible of owning that header file ?
Some compilers do support features like "post build", which runs a program of your choice after compiling, but that would be tricky if your program is built from multiple source files. Not all compilers do though.
For that reason, I wouldn't actually do this sort of thing via the compiler. I'd do it in the build script (e.g. makefile) or by configuring the build settings in your IDE.
Assuming you're using make or similar, add a target something like setversion (you pick the name) that runs a program which modifies a header file which specifies the components of your version number. So typing make setversion will update your version numbers.
Optionally, that target can also - after updating the version numbers - do a make clean (i.e. delete all object files and executables) and make all (to recompile and link everything).
I also suggest avoiding changing version numbers after each recompile. Imagine you're busily testing and debugging code, and go through several rebuild cycles. Do you really want the version number updated every time you recompile even one source file? It can be done that way if you choose, but will make every rebuild take longer (and, in projects with multiple source files) you will need to take care if you want to preserve capability for incremental builds.
When I build a project using a c++ compiler, can I make sure that the produced binary is not affected if there were no changes in source code? It looks like everytime I recompile my source, the binary's md5 checksum is affected. Is the time of compilation somehow affecting the binary produced? How can I produce compilation results which are repeatable?
One can disassemble the binaries and run md5 on the output
Example on MacOSX
otool -tV a.out | md5
ee2e724434a89fce96aa6b48621f7220
But, one misses out on the global data...(might be a parameter to include too)
I'm answering on the problem of md5 checking a binary...how you manage your sources and build system as others have written about is also a thing to look at
I suspect it'll heavily depend on your toolchain and OS. For example, if one of the executable headers contains a timestamp then you're always going to find the resulting MD5 is different.
What's the end result you're trying to achieve (ie why is it so important that they're identical)..?
You can't do a md5 checksum comparison for visual studio. For a normal Release version .exe file from visual studio there will be 3 locations that change with each recompile. 2 of them are timestamps and the third is a unique GUID that visual studio uses to match versions of the .exe with helper files to ensure they are in sync.
It might be possible to write a tool that will zero out the 3 changing fields, but I'm not sure how easy it would be to parse the file.
Also, if you are calling any .dlls, if I recall right, you will get more unique identifiers in the generated file.
The Debug version is a different story. I think there are many, many more differences.
Use an incremental build system - such as make to ensure you don't recompile your code if the source doesn't change.
It may be possible to get your compile to make identical binaries from the same source - or it may not - it depends on the compiler. Most will embed the current time in the generated binary somewhere.
I'm writing a function for an installer DLL to verify the Authenticode signature of EXE files already installed on the system.
The function needs to:
A) verify that the signature is valid.
B) verify that the signer is our organization.
Because this is in an installer, and because this needs to run on older Win2k installations, I don't want to rely on CAPICOM.dll, as it may not be on the target system.
The WinVerifyTrust API works great to solve (A).
I need to find a way to compare a known certificate (or properties therein) to the one that signed the EXE in question.
You should use CryptQueryObject.
This KB-article demonstrates the use: How To Get Information from Authenticode Signed Executables.
To the commenter that asked about how to do it without the Windows-APIs, I am not aware of any library that can do it, but the format is documented here: Windows Authenticode Portable Executable Signature Format
If the signature is valid, its certificate chain will contain your certificate. CertGetCertificateChain will get that chain.