OpenSSL build odd behaviour - build

Its kinda an odd story. I'm trying to build the openssl library. Well, it is not the first time I'm doing it and until now it was fine. however, I've desided to add a build configuration to TeamCity (actually the problem has nothing to do with TeamCity)to build all my third parties libraries on-demand. So, I created a step with common set of openssl build commands:
perl Configure VC-WIN64A
ms\do_win64a.bat
nmake /d /f ms\ntdll.mak
... and the build failed while assembling the very first asm file. So I ran same commands in VS2012 x64 command prompt and the build succedeed, as expected. It took a while untill I figured out that the problem is in output redirection. Teamcity redirects output from build step to the teamcity build log, so did I by adding ">1.txt" to "nmake /d /f ms\ntdll.mak" command, and the build failed. I would say it is somewhat unexpected reason for build to fail.
So far, I've tried:
Building the same from VS2010 command prompt with redirection,
failed.
Building the same from VS2010 command prompt without
redirection, succeded.
Building the same from VS2010/VS2012 command
prompt with redirection to file and forcing redirection to CON (nmake
/d /f ms\ntdll.mak >CON >1.txt) failed.
Googling for "nmake problems
when console output is redirected" - nothing
Googling for "openssl
build problems when console output is redirected" - nothing
My setup
- Win2008R2, virtual machine.
- VS2010, latest updates
- VS2012, Update 1
Actual error:
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\amd64\ml64.EXE"' : return code '0x1'
ml64.exe complains about the asm file stating it should end with "END" statement. It does, but...
Any ideas?

Ok, since the problem is in perl and writing files to stdout and then redirecting it you have to change following files to build x64 version:
crypto\aes\asm\aes-x86_64.pl
crypto\bn\asm\modexp512-x86_64.pl
crypto\bn\asm\x86_64-gf2m.pl
crypto\bn\asm\x86_64-mont.pl
crypto\bn\asm\x86_64-mont5.pl
crypto\md5\asm\md5-x86_64.pl
crypto\modes\asm\ghash-x86_64.pl
crypto\rc4\asm\rc4-md5-x86_64.pl
crypto\rc4\asm\rc4-x86_64.pl
crypto\sha\asm\sha1-x86_64.pl
crypto\sha\asm\sha512-x86_64.pl
crypto\whrlpool\asm\wp-x86_64.pl
crypto\x86_64cpuid.pl
engines\asm\e_padlock-x86_64.pl
you have to open these files and change all "open STDOUT "whateverfile"" to something like "open my $asmfile "whaeverfile"", then take care of string concatenation (only in x86_64cpuid.pl) and then print it to the file instead of STDOUT, something like "print $code" to "print $asm $code"
and of course, dont forget to close the file, in another words change the "close STDOUT" to "close $asmfile"
Obviously when building target other than x64 one will have to change another set of files.
Will try to communicate it to openssl maintainer.

Related

Unable to build GDAL Visual Studio project with "generate_vcxproj.bat"

I need to link GDAL library to a c++ project.
I use the instruction from here:
http://trac.osgeo.org/gdal/wiki/GeneratingVisualStudioProject
Download gdal231.zip from official web site. Unpack.
(this step may be omitted — result will be the same) Edit nmake.opt by adding this in the very top of the file:
MSVC_VER=1910
WIN64=1
(don't know if it's right; there is no thorough guide)
Run win cmd, type "generate_vcxproj.bat 15.0 64 gdal_vs2017", wait for finish.
Open gdal_vs2017.vcxproj with MSVS Community 15.8.4. Build solution.
Build failed with "fatal error C1083: Cannot open include file: 'stdio.h': No such file or directory".
Project settings look strange:
1
"VC++ Directories" seem normal, except $(VC_SourcePath) in "Library Directories":
2
What's wrong with it?
In order to build GDAL one doesn't need a Visual Studio project. Don't know what's wrong with "generate_vcxproj" script — it just does'nt work. But there is another way.
Unpack gdal231.zip.
Launch "x64 Native Tools Command Prompt for VS 2017" (or whatever version of VS command prompt that suits your needs). cd to "gdal-2.3.1\" dir.
Open nmake.opt file. In it's beginning there are some hints for usage of makefile.vc. Pick some arguments from there. I opt for x64 release build, so my call to nmake was like that: "nmake -f makefile.vc MSVC_VER=1910 WIN64=1". (1910 is a version of Visual Studio 2017).
Switch to "Native tools command prompt" and type "nmake -f makefile.vc MSVC_VER=1910 WIN64=1" there. Enter.
When it's finished, type "nmake -f makefile.vc MSVC_VER=1910 WIN64=1 install" and press Enter.
When that's finished too, type "nmake -f makefile.vc MSVC_VER=1910 WIN64=1 devinstall" and press Enter.
Done. The library is in C:\warmerda\bld\ folder.
I got the same problem when building with VS2017 Community Edition. I was able to build fine using VS2015 Community Edition.
Bottom line: when running the .bat, use the 2015 option.
Modify the generate_vcxproj.bat file using the information from here:
https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering
I modified mine to work with Visual Studio Community 2019 by modifying lines 55-70 to be this:
if "%_vcver_%"=="16.2" (
set _clver_=1922
set _vstoolset_=v1422
) else if "%_vcver_%"=="15.0" (
set _clver_=1910
set _vstoolset_=v141
) else if "%_vcver_%"=="14.0" (
set _clver_=1900
set _vstoolset_=v140
) else ( if "%_vcver_%"=="12.0" (
set _clver_=1800
set _vstoolset_=v120
) else (
echo Wrong value for parameter 1. See usage:
goto :usage
))

Visual Studio cross platform makefile project, command not found

I'm trying to build a cross platform project for Ubuntu. In my makefile I have the line
"PSPSDK=$(shell psp-config --pspsdk-path)"
which gives the error "psp-config: Command not found."
psp-config is in my path and running make from the Ubuntu system on the files that get copied over from Visual Studio works fine. It also works if I manually ssh into the Ubuntu system from windows and run the command from there.
Why can't it find the command when run through Visual Studio?
You should update PATH at the beginning of "~/.bashrc" file (and not at the end) because it starts with somethign like:
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
Also, if you add code before these lines, it will be invoked for every subshell execution, so it is better to add a guard for it as well so it is invoked only once per session:
if [ -z $HOME_OPT_PATH_SET ]; then
export PATH=$PATH:$HOME/opt
export HOME_OPT_PATH_SET=1
fi
# If not running interactively, don't do anything
...

Can build Mysql Connector C++ on vs 2017?

I tried to build mysql-connector-c++ from source to static library.
Referring the official spec here, I generated the Visual studio solution by the following command:
cmake -DMYSQL_DIR="E:\mysql-5.7.21-winx64" -DWITH_BOOST="E:\libs\boost_1_67_0_BUILDED" -DWITH_JDBC=ON -G "Visual Studio 15 2017 Win64" ../
Then, I opened the vs solution generated by cmake.Run build, got the two libs: mysqlcppconn8-static-mt.lib, mysqlcppconn-static-mt.lib, and copied them to my project.
When I built my project (also vs2017 version), I got the error:
mysqlcppconn-static-mt.lib(net_serv.obj) : error LNK2038: 检测到“_MSC_VER”的不匹配项: 值“1800”不匹配值“1900”(ActionProcessor.obj 中)
I am confused. They're all built in vs2017, and platform toolset are all set to "Visual Studio 2017 (v141)". I can't understand where 1800_MSC_VER has come from.
Here is what I did
Specs
Visual Studio 15, (2017)
x64
Static (.lib)
With static runtime (/MT)
Tools
CMAKE
Bison (In a folder without spaces)
Perl (ActiveState or Strawberry)
OpenSSL 1.0.2q.tar.gz
Add CMAKE, Bison and pearl to your system path
Check to see if you got all setup correctly
C:\>cmake --version
cmake version 3.9.6
C:\>bison --version
bison (GNU Bison) 2.4.1
C:\>m4 --version
m4 (GNU M4) 1.4.13
C:\>perl -version
This is perl 5, version 24, subversion 3 (v5.24.3)
Step 1: Build OpenSSL 1.0.x
Note: if you want to build OpenSSL 1.1.x steps will be a bit different
Extract to: C:\mysql-8.0.15\openssl-1.0.2q
Run: x64 Native Tools Command prompt for VS 2017
cd C:\mysql-8.0.15\openssl-1.0.2q
Release
perl Configure VC-WIN64A no-shared --openssldir=.\rel64
ms\do_win64a
nmake -f ms\nt.mak
nmake -f ms\nt.mak install
Debug
nmake -f ms\nt.mak clean
perl Configure debug-VC-WIN64A no-shared --openssldir=.\dbg64
ms\do_win64a
nmake -f ms\nt.mak
nmake -f ms\nt.mak install
Step 2: Build MySQL (Yes the actual database)
Extract to: C:\mysql-8.0.15
Run: x64 Native Tools Command prompt for VS 2017
cd C:\mysql-8.0.15\
cmake . -G "Visual Studio 15 2017 Win64" ^
-DWITH_BOOST=C:/mysql-8.0.15/boost ^
-DDOWNLOAD_BOOST=1 ^
-DBUILD_CONFIG=mysql_release ^
-DWITH_SSL=C:\mysql-8.0.15\openssl-1.0.2q\rel64 ^
-DLINK_STATIC_RUNTIME_LIBRARIES=1
Copy m4.exe to C:\mysql-8.0.15\sql\ this avoids any m4.exe errors during built
Open MySQL.sln and build for release x64
Copy binary_log_funcs.h and binary_log_types.h from mysql-8.0.15\libbinlogevents\export to mysql-8.0.15\include
The library should be located at:
Static release library: C:\mysql-8.0.15\archive_output_directory\Release\mysqlclient.lib
Step 3: Build MySQL Connector C++
Copy C:\mysql-8.0.15-dbg\archive_output_directory\Debug\mysqlclient.lib
to C:\mysql-8.0.15-dbg\lib\vs14
Run CMAKE GUI
Browse to the path of the MySQL Connector Project.
Click Configure, Select Visual studio 15 2017 Win64, native compilers
Tick BUILD_STATIC
Tick STATIC_MSVCRT
TICK WITH_JDBC
remove WIN_SSL_YASL from CONFIG_VARS
WITH_SSL = C:\mysql-8.0.15\openssl-1.0.2q\rel64
WITH_BOOST= C:\mysql-8.0.15\boost\boost_1_66_0
MYSQL_DIR=C:\mysql-8.0.15
Open MySQL_CONCPP.sln build for Release x64
The library is located at:
C:\mysql-connector-c++-8.0.15\jdbc\install\lib\mysqlcppconn-static-mt.lib
C:\mysql-connector-c++-8.0.15\Release\mysqlcppconn8-static-mt.lib
Step 4: Test it
Note: This test is for the JDBC style API. If you are not maintaining an existing application, use the new xdevapi API.
Create a new project and put the following in main.cpp
#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#define STATIC_CONCPP
#define CONCPP_BUILD_STATIC
#define CPPCONN_PUBLIC_FUNC
#include <driver.h>
#include <connection.h>
#include <statement.h>
#include <exception.h>
sql::Driver *mDriver;
sql::Connection *mConnection;
int main() {
mDriver = get_driver_instance(); //or use: new MySQL_Driver()
try {
mConnection = mDriver->connect("host", "username", "password");
mConnection->close();
}catch(sql::SQLException& ex) {
std::cout << ex.what();
}
delete mConnection;
return 0;
}
Set: Linker > Input > Additional Dependecies:
libeay32.lib;mysqlclient.lib;mysqlcppconn8-static-mt.lib;mysqlcppconn-static-mt.lib;ssleay32.lib;%(AdditionalDependencies)
Set: Configuration Properties > VC++ Directories
Include Directories.
Library Directories.
I don't even know where to start explaining to be honest. Building this was a massive pain.
Before you begin to read this, please ensure you have the following:
C:\boost
C:\OpenSSL-Win32
C:\OpenSSL-Win64
Because I was writing my own "wrapper" and trying to make UNICODE work properly, I was getting weird exceptions, and I was unable to debug anything because I downloaded the latest package which is built without debug information (https://dev.mysql.com/downloads/connector/cpp/)
This is the first time I try to build this, so I went to their Github repository and grabbed the latest "master": https://github.com/mysql/mysql-connector-cpp
BUT! This does not contain the "JDBC" sources, which is by the way the "native" connector as they call it. So I grabbed it from the branch: https://github.com/mysql/mysql-connector-cpp/tree/jdbc and then copied the files into "mysql-connector-cpp-master\jdbc".
OK. I opened "x64 Native Tools Command Prompt for VS 2017" and navigated to "mysql-connector-cpp-master". At this point I still had no idea what I am doing and how to build it, all I could understand from the docs and different articles here is to run:
cmake -DWITH_JDBC=ON --build .
This started to configure things and build OK, but it stopped here:
-- Searching for static libraries with the base name(s) "mysqlclient"
CMake Error at FindMySQL.cmake:524 (message):
Could not find "mysql.h" from searching "/usr/include/mysql
/usr/local/include/mysql /opt/mysql/mysql/include
/opt/mysql/mysql/include/mysql /usr/local/mysql/include
/usr/local/mysql/include/mysql C:\Program Files/MySQL/*/include
C:/MySQL/*/include"
Call Stack (most recent call first):
CMakeLists.txt:332 (INCLUDE)
OK. So what now... where is "mysql.h" ?! After searching... and searching, I found this: Missing mysql.h and trying to find mysql-devel
And I quote:
Grab the MySQL Community Server and install it on your system. It
contains a directory called include in which you find the mysql.h.
Ok, great we need more stuff. So I download: https://dev.mysql.com/downloads/mysql/ and run the command again.
Happy days, something happened but no libs yet. I did get however a VS 2017 solution called: "MYSQLCPPCONN.sln".
This contains VS 2017 configuration to build the library for Win32. And of course trying to build with this solution will give you A LOT of unresolved external symbols.
So I run again the cmake but with static option added:
cmake -DWITH_JDBC=ON -DBUILD_STATIC=ON --build .
This time when I open "MYSQLCPPCONN.sln" I can see another project "mysqlcppconn-static" and this builds fine without issues.
However, in my case I need x64. I just created a new configuration for x64 by copying the exiting configuration fron Win32.
So that's it, I have both Win32 and x64 "mysqlcppconn-static.lib".
Overall Conclusion at this time:
Quite hard to build, without experience you will spend some time...
time.
No proper documentation, their existing documentation is poorly
written and very confusing.
No UNICODE support, yes the projects compile as Multi-Byte
Character Set.
I will see if I can debug and find out why am I getting exceptions in the library when using this (from their documentation):
sql::ConnectOptionsMap connection_properties;
connection_properties["hostName"] = "";
connection_properties["port"] = "";
connection_properties["userName"] = "";
connection_properties["password"] = "";
connection_properties["schema"] = "";
connection_properties["OPT_CONNECT_TIMEOUT"] = 10;
connection_properties["CLIENT_MULTI_STATEMENTS"] = (true);
connection_properties["OPT_CHARSET_NAME"] = "utf8";
connection_properties["OPT_SET_CHARSET_NAME"] = "utf8";
I hope you can build your library with this information, and hopefully in the future it will become more easier and clear.
Good Luck.

LNK1181 error when compiling V8 engine on Win10

I'm following this guide on building V8 but I am hitting some issues on the compilation step. I am running Windows 10 x64. I am trying to compile with options to embed the engine also.
Running the following command:
ninja -C out.gn/x64.release
Gives me this error:
ninja: Entering directory `out.gn/x64.release'
[1/471] LINK mksnapshot.exe mksnapshot.exe.pdb
FAILED: mksnapshot.exe mksnapshot.exe.pdb
C:/Workspace/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False link.exe /nologo /OUT:./mksnapshot.exe /PDB:./mksnapshot.exe.pdb #./mksnapshot.exe.rsp
LINK : fatal error LNK1181: cannot open input file 'comdlg32.lib'
ninja: build stopped: subcommand failed.
Now I believe I have narrowed down the error to looking for the .lib files in the wrong directory. I have (had) multiple versions installed, so there were multiple folders in my Windows Kit install.
Windows Kits/10/Lib/10.0.16299.0
Windows Kits/10/Lib/10.0.15xxx.0
If I dragged and dropped the comdlg32.lib file from 10.0.16299.0 into the 10.0.15xxx.0 directory then the error changed to a LNK1181 error with a different input file. I did this a few times but I was unsure if this was going to cause issues with different versions and there was probably going to be a lot.
I uninstalled the 10.0.15xxx.0 version which left behind the folder I mentioned, so I removed that and after doing so I have started getting the LNK1181 error with a different input file (advapi32.lib I assume the very first file it can't find). This is how I came to the conclusion about the path being incorrect.
So I have tried a few things to change the path (I hoped just uninstalling the old version would fix it) such as:
Uninstalling the old version.
Going through registry entries to see if I can find an install path or something using that path, which I didn't. I did notice that there was still installation and data in the registry for the 10.0.15xxx.0 install, I might try deleting that from the registry directly as a last resort?
I have tried to explicitly set the path by setting <TargetUniversalCRTVersion>10.0.16299.0</TargetUniversalCRTVersion> in this file: C:\Program Files (x86)\Windows Kits\10\DesignTime\CommonConfiguration\Neutral\uCRT.props
I have never used Ninja before so I tried looking for a way to set some kind of lib-path in the command but couldn't really find anything.
I looked through the python scripts being executed to try and locate something to do with the libs path but couldn't see anything.
I would be grateful for any help and suggestions. Thanks.
You can try to compile v8 using Visual Studio as explained here: https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md#using-the-visual-studio-ide
By running the following commands:
$ gn gen --ide=vs out.gn/x64.release
$ cd out.gn/x64.release
$ msbuild all.sln
You can see a full example here: https://github.com/phpv8/v8js/issues/272#issuecomment-262848754
Apparently this method is not officially supported anymore, but I had the same problem as you have and this solved the issue for me.
Note that after this I had another issue, the unit tests failed to be compiled due to a linking error, but I had the necessary libraries to use v8. So there may be deeper problem that is causing all of this that I'm missing.
Edit:
Also, you could try to set the following parameters with gn args:
visual_studio_path = "..."
visual_studio_version = "2017"
wdk_path = "..."
windows_sdk_path = "C:\Program Files (x86)\Windows Kits\10"
To set those parameters, do:
gn args out.gn/x64.release
This will open a text editor where you can write the extra parameters you are interested in.
To see the full list of parameters you can specify:
gn args --list out.gn/x64.release
I was following this guide https://medium.com/dailyjs/how-to-build-v8-on-windows-and-not-go-mad-6347c69aacd4 and also ran into the error
LINK1181: cannot open input file 'advapi32.lib'
I'm pretty sure it was because I had the wrong versions of the Windows 10 SDK. Similar to you I had versions:
Windows Kits/10/Lib/10.0.10240.0
Windows Kits/10/Lib/10.0.16299.0
But according to https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md#Setting-up-Windows (Which I think is relevant) you need version 10.0.15063.0
After installing version 10.0.15063.0 (with the visual studio installer) to
Windows Kits/10/Lib/10.0.15063.0
I was able to continue with the build.

InstallShield creates MSI even though build has errors

When I'm compiling ism project to create MSI, its still creates the MSI even though I have build errors.
The reason I need it NOT to be created is for build verification.
Instead of checking the build log for errors, I will just check the existence of the MSI.
Does anybody know how can I achieve that?
EDIT:
I'm using ISCmdBld tool to build MSIs. This is the command line I'm running to build where the environment variables are being set before running this command:
IsCmdBld -p "%FULL_PROJECT_FILENAME%" -a %BUILDMODE% -r %PRODUCT% -o "%MMSEARCHPATH%" | tee /A "%FULL_PROJECT_LOG_FILENAME%"
If you are compiling using IsCmdBld.exe, you should add the -x option, so that the build is stopped if an error occurs.
You also can use it combined with -w, which makes each warning becomes considered as an error (and thus, each warning encountered also stops the build).
More information about IsCmdBld.exe : http://helpnet.installshield.com/installshield16helplib/ISCmdBldParam.htm
I hope this helps.
Your build automation should check the exit code from ISCmdBld.exe. If the exit code is a failure, don't archive the output.