I am trying to use OpenSSL but I am stuck on the step of compiling. The OpenSSL project has very unfriendly (bad) documentation.
Is there any actual help how to build the latest OpenSSL version on Windows with Visual Studio 2017?
I didn't find any helpful information on the official OpenSSL site. Yes, there are a lot of posts on the Internet about OpenSSL compilation, but all of them are obsolete.
I've not used VS2017 but previous versions. I imagine it is much the same. Note the instructions below are for OpenSSL 1.1.0 or above. They do not work for OpenSSL 1.0.2. In brief the steps are:
Install Perl (either ActiveState or Strawberry)
[EDIT, see my (kritzel_sw) comment below: I would strongly recommend to use Strawberry)]
Install NASM
Make sure both Perl and NASM are on your %PATH%
Fire up a Visual Studio Developer Command Prompt with administrative privileges (make sure you use the 32-bit one if you are building 32-bit OpenSSL, or the 64-bit one if you are building 64-bit OpenSSL)
From the root of the OpenSSL source directory enter perl Configure VC-WIN32, if you want 32-bit OpenSSL or perl Configure VC-WIN64A if you want 64-bit OpenSSL
Enter nmake
Enter nmake test
Enter nmake install
[EDIT, unless you change the target directory in the configuration, nmake install needs administrator privileges. So the VC command prompt must be started as administrator for this final step]
If anything goes wrong at any stage, check the INSTALL file and the NOTES.WIN file.
Modified version of The Quantum Physicist python script
It can compile OpenSSL 1.0.x or OpenSSL 1.1.x
It can compile with multiple version of Visual Studio 2017/2019 included.
1) Create the file: CompileOpenSSL.py
import os
import os.path
from subprocess import call
import shutil
import sys
import re
import argparse
# args
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--filename", help="First argument must be the tar.gz file of OpenSSL source", required=True)
parser.add_argument("-a", "--arch", help="Second argument must be x86 or amd64", required=True)
parser.add_argument("-v", "--vs_version", help="Visual Studio version (eg:90, 140, 150)", required=True)
parser.set_defaults(writeVersionInfos=False)
args = parser.parse_args()
compile_flags = "-no-asm"
#compile_flags = "-no-asm -no-shared"
openssl_32_flag = "VC-WIN32"
openssl_64_flag = "VC-WIN64A"
working_dir = os.getcwd()
dirname = args.filename.replace(".tar.gz","")
src_32_suffix = "_" + "vs" + args.vs_version + "_32"
src_64_suffix = "_" + "vs" + args.vs_version + "_64"
vs_tools_env_var = "VS" + args.vs_version + "COMNTOOLS"
if args.arch != "x86" and args.arch != "amd64":
print("Second argument must be x86 or amd64")
exit(1)
if not bool(re.match("(openssl-){1}(\d)+(.)(\d)+(.)(\d)+(\w)+(.tar.gz)",args.filename)):
print("The file given doesn't seem to be an openssl source file. It must be in the form: openssl-x.y.zw.tar.gz")
exit(1)
call("7z x -y " + args.filename) #extract the .gz file
dirname_src_32 = dirname + src_32_suffix
dirname_src_64 = dirname + src_64_suffix
dirname_bin_32 = dirname + src_32_suffix + "_build"
dirname_bin_64 = dirname + src_64_suffix + "_build"
openssl_tar_file = args.filename[0:-3]
if args.arch == "x86":
#delete previous directories
shutil.rmtree(os.getcwd()+'/'+dirname, ignore_errors=True)
shutil.rmtree(os.getcwd()+'/'+dirname_src_32, ignore_errors=True)
#extract tar file for 32
call("7z x -y " + openssl_tar_file)
os.rename(dirname, dirname_src_32)
#Compile 32
os.chdir(dirname_src_32)
print("perl Configure " + openssl_32_flag + " --prefix=" + os.path.join(working_dir,dirname_bin_32) + " " + compile_flags)
call("perl Configure " + openssl_32_flag + " --prefix=" + os.path.join(working_dir,dirname_bin_32) + " " + compile_flags,shell=True)
if( os.path.exists("ms/do_ms.bat") ):
call("ms\do_ms.bat",shell=True)
print(os.getcwd())
call("nmake -f ms/ntdll.mak",shell=True)
call("nmake -f ms/ntdll.mak install",shell=True)
else:
call("nmake",shell=True)
call("nmake test",shell=True)
call("nmake install",shell=True)
print("32-bit compilation complete.")
#Go back to base dir
os.chdir(working_dir)
################
if args.arch == "amd64":
#delete previous directories
shutil.rmtree(os.getcwd()+'/'+dirname, ignore_errors=True)
shutil.rmtree(os.getcwd()+'/'+dirname_src_64, ignore_errors=True)
#extract for 64
call("7z x -y " + openssl_tar_file)
os.rename(dirname, dirname_src_64)
#Compile 64
os.chdir(dirname_src_64)
call("perl Configure " + openssl_64_flag + " --prefix=" + os.path.join(working_dir,dirname_bin_64) + " " + compile_flags,shell=True)
if( os.path.exists("ms\do_ms.bat") ):
call("ms\do_win64a.bat",shell=True)
call("nmake -f ms/ntdll.mak",shell=True)
call("nmake -f ms/ntdll.mak install",shell=True)
else:
call("nmake",shell=True)
call("nmake test",shell=True)
call("nmake install",shell=True)
print("64-bit compilation complete.")
#Go back to base dir
os.chdir(working_dir)
################
os.remove(openssl_tar_file)
2) Create the file: CompileOpenSSL_vs.cmd
ECHO --------------------------------------
ECHO Require Python, 7Zip, PERL and NASM in PATH
ECHO --------------------------------------
Rem ------------------------------------------------------
Rem TO CONFIGURE -----------------------------------------
Rem ------------------------------------------------------
Rem SET YOUR LOCAL PATHS-----------------------------------------
SET PATH=C:\Program Files (x86)\7-Zip;C:\Perl64\bin;M:\Backup\Coders\_tools\7-Zip\;%PATH%
Rem SET YOUR OPENSSL ARCHIVE-----------------------------------------
REM SET FILENAME=openssl-1.0.2r.tar.gz
SET FILENAME=openssl-1.1.1b.tar.gz
Rem SET THE VERSION OF YOUR VISUAL STUDIO-----------------------------------------
SET VSVERSION=%1
Rem ------------------------------------------------------
Rem COMPILATION LAUNCH -----------------------------------
Rem ------------------------------------------------------
Rem UTILS PATH-----
SET VSCOMNTOOLSNAME=VS%VSVERSION%COMNTOOLS
Rem Pick the good path for Visual Studio-----------------------------------------
IF %VSVERSION% GEQ 150 (
Echo DO NOT FORGET TO ADD A SYSTEM VARIABLE %VSCOMNTOOLSNAME% - like: "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\"
SET VCVARPATH="%%%VSCOMNTOOLSNAME%%%..\..\VC\Auxiliary\Build\vcvarsall.bat"
) ELSE (
SET VCVARPATH="%%%VSCOMNTOOLSNAME%%%..\..\VC\vcvarsall.bat"
)
Rem Set env -----------------------------------------
#pushd "%~dp0"
call %VCVARPATH% %2
#popd
Rem ------------------------------------------------------
Rem TEST APP EXIST -----------------------------------
Rem ------------------------------------------------------
where /q 7z.exe
IF ERRORLEVEL 1 (
ECHO The application "7z.exe" is missing. Ensure to add/install it to the PATH in beginning of this script, check SET PATH
PAUSE
EXIT /B
)
where /q perl.exe
IF ERRORLEVEL 1 (
ECHO The application "perl.exe" is missing. Ensure to add/install it to the PATH in beginning of this script, check SET PATH
PAUSE
EXIT /B
)
where /q nmake.exe
IF ERRORLEVEL 1 (
ECHO The application "nmake.exe" is missing. Ensure to add/install it to the PATH in beginning of this script, check SET PATH
PAUSE
EXIT /B
)
where /q py.exe
IF ERRORLEVEL 1 (
ECHO The application "py.exe" [shortcut of python] is missing. Ensure to add/install it to the PATH in beginning of this script, check SET PATH
PAUSE
EXIT /B
)
Rem Launch compilation -----------------------------------------
py CompileOpenSSL.py -f %FILENAME% -a %2 -v %VSVERSION%
PAUSE
3) Launch compilation from command line (Outside Visual Studio)
eg:
CompileOpenSSL_vs.cmd 150 x86
CompileOpenSSL_vs.cmd 150 amd64
CompileOpenSSL_vs.cmd 90 x86
For OpenSSL 1.0.2, I wrote a Python script that does the building for me. I have this habit of making these scripts, as I don't like to reinvent the wheel everytime I need to build something.
The script is made for OpenSSL 1.0.2. Probably the changes are minimal for OpenSSL 1.1.0.
Here's the script:
import os
from subprocess import call
import sys
import re
vs_version = "140"
compile_flags = "-no-asm -no-shared"
openssl_32_flag = "VC-WIN32"
openssl_64_flag = "VC-WIN64A"
src_32_suffix = "_" + "vs" + vs_version + "_32"
src_64_suffix = "_" + "vs" + vs_version + "_64"
vs_tools_env_var = "VS" + vs_version + "COMNTOOLS"
if len(sys.argv) < 2:
print("First argument must be the tar.gz file of OpenSSL source")
exit(1)
if len(sys.argv) < 3:
print("Second argument must be 32 or 64")
exit(1)
filename = sys.argv[1]
dirname = filename.replace(".tar.gz","")
working_dir = os.getcwd()
arch = sys.argv[2]
if arch != "32" and arch != "64":
print("Second argument must be 32 or 64")
exit(1)
if not bool(re.match("(openssl-){1}(\d)+(.)(\d)+(.)(\d)+(\w)+(.tar.gz)",filename)):
print("The file given doesn't seem to be an openssl source file. It must be in the form: openssl-x.y.zw.tar.gz")
exit(1)
call("7z x " + filename) #extract the .gz file
dirname_src_32 = dirname + src_32_suffix
dirname_src_64 = dirname + src_64_suffix
dirname_bin_32 = dirname + src_32_suffix + "_build"
dirname_bin_64 = dirname + src_64_suffix + "_build"
openssl_tar_file = filename[0:-3]
if arch == "32":
#extract tar file for 32
call("7z x " + openssl_tar_file)
os.rename(dirname, dirname_src_32)
#Compile 32
os.chdir(dirname_src_32)
call("perl Configure " + openssl_32_flag + " --prefix=" + os.path.join(working_dir,dirname_bin_32) + " " + compile_flags,shell=True)
call(r"ms\do_ms.bat",shell=True)
call(r"nmake -f ms\nt.mak",shell=True)
call(r"nmake -f ms\nt.mak instalL",shell=True)
print("32-bit compilation complete.")
#Go back to base dir
os.chdir(working_dir)
################
if arch == "64":
#extract for 64
call("7z x " + openssl_tar_file)
os.rename(dirname, dirname_src_64)
#Compile 64
os.chdir(dirname_src_64)
call("perl Configure " + openssl_64_flag + " --prefix=" + os.path.join(working_dir,dirname_bin_64) + " " + compile_flags,shell=True)
call(r"ms\do_ms.bat",shell=True)
call(r"nmake -f ms\nt.mak",shell=True)
call(r"nmake -f ms\nt.mak instalL",shell=True)
print("64-bit compilation complete.")
#Go back to base dir
os.chdir(working_dir)
################
os.remove(openssl_tar_file)
Option 1: Save the script to CompileOpenSSL.py, and download the OpenSSL source file that is expected to have the name format openssl-1.X.Y.tar.gz. Now assuming that 7zip and perl are accessible from the global scope on your command prompt and you have the correct MSVC variables loaded (with e.g. vsvars32.bat, or starting the right terminal), run the following:
python CompileOpenSSL.py openssl-1.X.Y.tar.gz 32
If you're using MSVC 32-bit, or
python CompileOpenSSL.py openssl-1.X.Y.tar.gz 64
for MSVC 64-bit.
Option 2: Do what the script does manually. The script simply extracts the archive, configures the sources and runs do_ms.bat then nmake. Follow the source and it'll work.
Good luck!
go into ssl directory using visual studio cmd and add perl and nasm to system path then type:
perl Configure --openssldir=D:OpenSSLdirectory VC-WIN32
ms\do_ms.bat
nmake -f ms\ntdll.mak
nmake -f ms\ntdll.mak install
( enjoy. )
Related
Here is my building process
I open mingw32 from the x64 Native Tools Command Prompt for VS 2022
then in the mingw32 shell:
# cd /
# ./c/Program\ Files/Microsoft\ Visual\ Studio/2022/Community/VC/Auxiliary/Build/vcvars32.bat
# cd ~
# pacman -Sy diffutils git make gcc yasm pkg-config --noconfirm
# git clone --depth 1 https://git.ffmpeg.org/ffmpeg.git ffmpeg
# git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git nv-codec-headers
# cd nv-codec-headers/
# make PREFIX=/usr/local
# make install PREFIX=/usr/local
# cd ..
# mkdir nv_sdk
# cp -r /c/Program\ Files/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v11.7/lib/Win32/* nv_sdk
# cp -r /c/Program\ Files/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v11.7/include/* nv_sdk
# export PATH="/c/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.32.31326/bin/Hostx86/x86/":$PATH
# export PATH="/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/bin/":$PATH
# ./configure --disable-everything --enable-decoder=h264 --enable-decoder=hevc --enable-cross-compile --disable-avdevice --disable-swresample --disable-postproc --disable-avfilter --target-os=mingw32 --enable-cuda-nvcc --enable-nonfree --toolchain=msvc --extra-cflags=-I../nv_sdk --extra-ldflags=" -m32 -L../nv_sdk" --enable-shared --shlibdir=SHARED_LIBS --arch=x86_32 --enable-runtime-cpudetect --enable-w32threads
# make -j8
# make install
first I have a bunch of warnings during the making looking like that:
libavutil/opt.c(1075): warning C4133: 'fonction' : types incompatibles - de 'AVPixelFormat *' à 'int *'
And finally the make install returns :
EXTERN_PREFIX="_" AR="lib.exe" NM="dumpbin.exe -symbols" ./compat/windows/makedef libavutil/libavutil.ver libavutil/adler32.o libavutil/aes.o libavutil/aes_ctr.o libavutil/audio_fifo.o libavutil/avsscanf.o libavutil/avstring.o libavutil/base64.o libavutil/blowfish.o libavutil/bprint.o libavutil/buffer.o libavutil/camellia.o libavutil/cast5.o libavutil/channel_layout.o libavutil/color_utils.o libavutil/cpu.o libavutil/crc.o libavutil/des.o libavutil/detection_bbox.o libavutil/dict.o libavutil/display.o libavutil/dovi_meta.o libavutil/downmix_info.o libavutil/encryption_info.o libavutil/error.o libavutil/eval.o libavutil/fifo.o libavutil/file.o libavutil/file_open.o libavutil/film_grain_params.o libavutil/fixed_dsp.o libavutil/float_dsp.o libavutil/frame.o libavutil/hash.o libavutil/hdr_dynamic_metadata.o libavutil/hdr_dynamic_vivid_metadata.o libavutil/hmac.o libavutil/hwcontext.o libavutil/hwcontext_d3d11va.o libavutil/hwcontext_dxva2.o libavutil/imgutils.o libavutil/integer.o libavutil/intmath.o libavutil/lfg.o libavutil/lls.o libavutil/log.o libavutil/log2_tab.o libavutil/lzo.o libavutil/mastering_display_metadata.o libavutil/mathematics.o libavutil/md5.o libavutil/mem.o libavutil/murmur3.o libavutil/opt.o libavutil/parseutils.o libavutil/pixdesc.o libavutil/pixelutils.o libavutil/random_seed.o libavutil/rational.o libavutil/rc4.o libavutil/reverse.o libavutil/ripemd.o libavutil/samplefmt.o libavutil/sha.o libavutil/sha512.o libavutil/slicethread.o libavutil/spherical.o libavutil/stereo3d.o libavutil/tea.o libavutil/threadmessage.o libavutil/time.o libavutil/timecode.o libavutil/tree.o libavutil/twofish.o libavutil/tx.o libavutil/tx_double.o libavutil/tx_float.o libavutil/tx_int32.o libavutil/utils.o libavutil/version.o libavutil/video_enc_params.o libavutil/x86/cpu.o libavutil/x86/cpuid.o libavutil/x86/fixed_dsp.o libavutil/x86/fixed_dsp_init.o libavutil/x86/float_dsp.o libavutil/x86/float_dsp_init.o libavutil/x86/imgutils.o libavutil/x86/imgutils_init.o libavutil/x86/lls.o libavutil/x86/lls_init.o libavutil/x86/tx_float.o libavutil/x86/tx_float_init.o libavutil/xga_font_data.o libavutil/xtea.o > libavutil/avutil-57.def
Could not create temporary library.
make: *** [ffbuild/library.mak:118: libavutil/avutil-57.dll] Error 1
What am I doing wrong ?
shall I install others packets from pacman?
I open mingw32 from the x64 Native Tools Command Prompt for VS 2022
maybe you should open it in X86 Native Tools Command Prompt for VS 2022
I have a variable declared in .bzl file example: VERSION_NUMBER = "00".
I want to override this variable from command line when i build different version of the project.
example: bazel build target --sunbversion_number= "99"
I want to change this variable because it is invoked in some function to create the name of the output paths.
exammple: for version "00" the outputfile will be: name_00.extension
and for version "99" the outputfile will be: name_99.extension
This is my example:
in .bzl file i declared:
SUBVERSION_NUMBER = "99"
and a fucntion that return a name of the file regarding to the SUBVERSION_NUMBER
def get_name(SUBVERSION_NUMBER):
return "test-"+SUBVERSION_NUMBER
OUTPUT_NAME = get_name("99")
Then my genrule():
genrule(name = "test",
srcs = [srcs],
outs = [OUTPUT_NAME+".tek"],
cmd = "cmd to generate the file" )
when i build this rule i get the output file test-99.tek
what i want is when i run bazel build test --//version=01 or any other suggested solution, i want to get output test-01.tek
Thanks
There is no way to get values from the command line into a bzl file like that, but there are a few options, depending on exactly what you want to do.
One way is to use "stamping" to set the version, and then put the versioned file in a zip file with a known name, e.g. using a genrule. Build stamping is usually used for embedding a version number and other information into the output files themselves, but can be used here too. A zip file is necessary because the output file name has to be known at load time (i.e. before any configuration data from the command line is processed at analysis time).
Something like this:
# out.txt is the original unversioned file
genrule(
name = "gen_out",
outs = ["out.txt"],
cmd = "echo foo > $#",
)
genrule(
name = "gen_versioned_out",
outs = ["out_versioned.zip"],
srcs = [":out.txt"],
tools = ["#bazel_tools//tools/zip:zipper"],
stamp = True,
cmd = """
# bazel-out/stable-status.txt is created when stamp = True
# Value of BUILD_EMBED_LABEL key comes from --embed_label on the command line
version="$$(grep BUILD_EMBED_LABEL bazel-out/stable-status.txt | cut -d ' ' -f 2)"
# Set a reasonable default if --embed_label was not specified
if [ -z "$$version" ]; then version="0"; fi
file="$$(basename $<)"
name="$${file%%.*}"
ext="$${file#*.}"
versioned_file="$${name}_$${version}.$${ext}"
# zipper allows specifying the name of the file in the zip directly, unlike the
# regular zip tool.
# c = create
# $# = output file from "outs"
# $< = input file from "srcs"
# $$versioned_file=$< = "put this file in to the archive with this name"
$(location #bazel_tools//tools/zip:zipper) c "$#" "$$versioned_file=$<"
""",
)
Then:
$ bazel build out_versioned.zip --embed_label=99
INFO: Analyzed target //:out_versioned.zip (7 packages loaded, 19 targets configured).
INFO: Found 1 target...
Target //:out_versioned.zip up-to-date:
bazel-bin/out_versioned.zip
INFO: Elapsed time: 0.340s, Critical Path: 0.10s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
$ unzip -l bazel-bin/out_versioned.zip
Archive: bazel-bin/out_versioned.zip
Length Date Time Name
--------- ---------- ----- ----
4 2010-01-01 00:00 out_99.txt
--------- -------
4 1 file
So that results in a zip file with a known name containing the versioned file.
Another approach is to use "User-defined build settings":
https://docs.bazel.build/versions/master/skylark/config.html#user-defined-build-settings
Something like this:
defs.bzl:
def _version_file_impl(ctx):
version = ctx.attr._version[VersionProvider].version
name, extension = ctx.file.file.basename.rsplit(".", 1)
versioned_file = ctx.actions.declare_file(
"%s_%s.%s" % (name, version, extension))
copy_args = ctx.actions.args()
copy_args.add_all([ctx.file.file, versioned_file])
ctx.actions.run_shell(
inputs = [ctx.file.file],
outputs = [versioned_file],
command = 'cp "$1" "$2"',
arguments = [copy_args])
return DefaultInfo(files = depset([versioned_file]))
version_file = rule(
implementation = _version_file_impl,
attrs = {
"file": attr.label(mandatory = True, allow_single_file = True),
"_version": attr.label(default = "//:version"),
}
)
VersionProvider = provider(fields = ["version"])
def _version_flag_impl(ctx):
return VersionProvider(version = ctx.build_setting_value)
version_flag = rule(
implementation = _version_flag_impl,
build_setting = config.int(flag = True),
)
BUILD:
load(":defs.bzl", "version_flag", "version_file")
version_flag(
name = "version",
build_setting_default = 0,
)
genrule(
name = "gen_out",
outs = ["out.txt"],
cmd = "echo foo > $#",
)
version_file(
name = "versioned_out",
file = ":out.txt",
)
Then:
$ bazel build :versioned_out --//:version=99
INFO: Analyzed target //:versioned_out (5 packages loaded, 10 targets configured).
INFO: Found 1 target...
Target //:versioned_out up-to-date:
bazel-bin/out_99.txt
INFO: Elapsed time: 0.322s, Critical Path: 0.06s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
So that results in a file with the version in its name. There's no label to refer to the versioned file itself though, so bazel build :out_99.txt nor srcs = [":out_99.txt"] will work, you have to go through the versioned_out target.
Update:
Here's a version that can version multiple outputs:
defs.bzl:
def _versioned_files_impl(ctx):
version = ctx.attr._version[VersionProvider].version
versioned_files = []
for f in ctx.attr.src.files.to_list():
name, extension = f.basename.rsplit(".", 1)
versioned_file = ctx.actions.declare_file(
"%s_%s.%s" % (name, version, extension))
versioned_files.append(versioned_file)
copy_args = ctx.actions.args()
copy_args.add_all([f, versioned_file])
ctx.actions.run_shell(
inputs = [f],
outputs = [versioned_file],
command = 'cp "$1" "$2"',
arguments = [copy_args])
return DefaultInfo(files = depset(versioned_files))
versioned_files = rule(
implementation = _versioned_files_impl,
attrs = {
"src": attr.label(mandatory = True),
"_version": attr.label(default = "//:version"),
}
)
VersionProvider = provider(fields = ["version"])
def _version_flag_impl(ctx):
return VersionProvider(version = ctx.build_setting_value)
version_flag = rule(
implementation = _version_flag_impl,
build_setting = config.int(flag = True),
)
BUILD:
load(":defs.bzl", "version_flag", "versioned_files")
version_flag(
name = "version",
build_setting_default = 0,
)
genrule(
name = "gen_out",
outs = ["foo.txt", "bar.txt", "baz.txt"],
cmd = """
echo foo > $(location foo.txt)
echo bar > $(location bar.txt)
echo baz > $(location baz.txt)
""",
)
versioned_files(
name = "versioned_files",
src = ":gen_out",
)
usage:
$ bazel build versioned_files --//:version=123
INFO: Analyzed target //:versioned_files (5 packages loaded, 9 targets configured).
INFO: Found 1 target...
Target //:versioned_files up-to-date:
bazel-bin/foo_123.txt
bazel-bin/bar_123.txt
bazel-bin/baz_123.txt
INFO: Elapsed time: 0.491s, Critical Path: 0.06s
INFO: 5 processes: 1 internal, 4 linux-sandbox.
INFO: Build completed successfully, 5 total actions
Update:
An example of putting the version in a cc target's define:
BUILD:
cc_binary(
name = "main",
srcs = ["main.cc"],
defines = ["VERSION=\\\"$(VERSION)\\\""],
)
main.cc:
#include <iostream>
#ifndef VERSION
#define VERSION "0.0.0"
#endif
int main() {
std::cout << "version: " << VERSION << std::endl;
return 0;
}
build and run:
$ bazel run main --define=VERSION=1.2.3
INFO: Analyzed target //:main (15 packages loaded, 52 targets configured).
INFO: Found 1 target...
Target //:main up-to-date:
bazel-bin/main
INFO: Elapsed time: 0.524s, Critical Path: 0.26s
INFO: 6 processes: 4 internal, 2 linux-sandbox.
INFO: Build completed successfully, 6 total actions
INFO: Build completed successfully, 6 total actions
version: 1.2.3
Combining the above methods, you would have to specify both --//:version=1.2.3 and --define=VERSION=1.2.3 on the command line. There's a way to have only --//:version, but it would require another Starlark rule like versioned_files which either
generates a file with the version in it that goes in the data attribute and the program reads at runtime, or
a Starlark rule which generates a C++ file with the version in it, which then gets put in the sources of a cc_library, which the rest of your program can depend on and use at compile time.
These approaches will probably require refactoring your program.
I am trying to automate the process of sending my temporary Amazon AWS keys as environment variables to a Docker image using Windows. I have a file, credentials.txt that contains my AWS credentials (the 3 ids are always the same, but the string values change regularly). I am using Windows command prompt.
Input:
(includes 2 empty lines at end) credentials.txt:
[default]
aws_access_key_id = STR/+ing1
aws_secret_access_key = STR/+ing2
aws_session_token = STR/+ing3
Desired output:
I need to issue the following command in order to run a Docker image (substituting the strings with the actual strings):
docker run -e AWS_ACCESS_KEY_ID=STR/+ing1 -e AWS_SECRET_ACCESS_KEY=STR/+ing2 -e AWS_SESSION_TOKEN=STR/+ing3 my-aws-container
My idea is to try to use regex on credentials.txt to convert it to:
SET aws_access_key_id=STR/+ing1
SET aws_secret_access_key=STR/+ing2
SET aws_session_token=STR/+ing3
And then run:
docker run -e AWS_ACCESS_KEY_ID=%aws_access_key_id% -e AWS_SECRET_ACCESS_KEY=%aws_secret_access_key% -e AWS_SESSION_TOKEN=%aws_session_token% my-aws-container
Does anyone have any advice on how to achieve this?
You can parse your credentials.txt with a for /f loop to set the variables (effectively removing the spaces):
for /f "tokens=1,3" %%a in ('type credentials.txt ^| find "="') do set "%%a=%%b"
and then run the last code line from your question:
docker run -e AWS_ACCESS_KEY_ID=%aws_access_key_id% -e AWS_SECRET_ACCESS_KEY=%aws_secret_access_key% -e AWS_SESSION_TOKEN=%aws_session_token% my-aws-container
Note: the values should not contain spaces or commas.
I've had a go in python that seems to work. Someone else may have a better answer.
I create the python file:
docker_run.py
import re
import os
myfile = 'C:/fullpath/credentials'
with open(myfile,'r') as f:
mystr = f.read()
vals = re.findall('=[\s]*([^\n]+)',mystr)
keys = ['AWS_ACCESS_KEY_ID','AWS_SECRET_ACCESS_KEY','AWS_SESSION_TOKEN']
environment_vars = ''.join([' -e ' + k + '=' + v for k,v in zip(keys,vals)])
cmd = 'docker run'+environment_vars+' my-aws-container'
os.system(cmd)
Then from command prompt I run:
python docker_run.py
This succeeds in running docker
(note: I tried using exec() in the final line rather than os.system(), but got the error "SyntaxError: invalid syntax")
I am attempting to build the ACE library for Mingw GCC 64 bit on Windows. The instructions here state the following:
Install the MinGW tools (including the MinGW Development toolkit) into a common directory, say c:/mingw.
Install the MSYS tools into a common directory, say c:/msys.
Open a MSYS shell. Set your PATH environment variable so your MinGW's bin directory is first:
% export PATH=/c/mingw/bin:$PATH
Add an ACE_ROOT environment variable pointing to the root of your ACE wrappers source tree:
% export ACE_ROOT=/c/work/mingw/ACE_wrappers
From now on, we will refer to the root directory of the ACE source tree as $ACE_ROOT.
Create a file called config.h in the $ACE_ROOT/ace directory that contains:
#include "ace/config-win32.h"
Create a file called platform_macros.GNU in the $ACE_ROOT/include/makeinclude directory containing:
include $(ACE_ROOT)/include/makeinclude/platform_mingw32.GNU
In the above text, don't replace $(ACE_ROOT) with the actual directory, GNU make will take the value from the environment variable you defined previously.
If you lack Winsock 2, add the line
winsock2 = 0
before the previous one.
If you want to install ACE (using "make install") and want all the .pc files generated, set the installation prefix in platform_macros.GNU.
INSTALL_PREFIX=/c/ACE
Headers will be installed to $INSTALL_PREFIX/include, documentation and build system files to $INSTALL_PREFIX/share and libraries to $INSTALL_PREFIX/lib. With INSTALL_PREFIX set, RPATH will be enabled. To disable RPATH (for example, if $INSTALL_PREFIX/$INSTALL_LIB is already a system-known location for shared libraries), set the make macro install_rpath to 0 by adding install_rpath=0 to platform_macros.GNU.
Issue here:
In the MSYS shell, change to the $ACE_ROOT/ace directory and run make:
% cd $ACE_ROOT/ace
% make
Now I noticed that there is no MakeFile in ACE_ROOT/ace which is C:\mingw64\Other\ACE_wrappers\ace
I downloaded my ACE stuff from here.
Any suggestions on what I might be doing wrong ? did I download something wrong ?
You seem to have downloaded the source only distribution, please download the full package, that includes also the GNU makefiles, see http://download.dre.vanderbilt.edu/
ACE comes in the full version with GNUmakefile-s,
In MSYS you give make -f GNUmakefile
EDIT 1
Though building 64-bit binaries is supported for numerous platforms and compilers, the ACE team did not provide it for MINGW. There is something to do ...
EDIT 2
Following script should do the configuration for 64-bit binaries in MingW-64
#! /bin/bash
#
# Configure ACE/TAO for 32/64 bit build with MINGW64
#
# Precondition:
# This script is located in the parent folder of ACE_Wrappers
# File access permissions in ACE_Wrappers allow editing of files (sed):
# Easyest, delivered full ACE/TAO ZIP was extracted using Windows Explorer.
# When extracting with 7z, it will correctly preserve access rights and
# they need to be granted for the user, explicitly
#
# Postcondition:
# ACE is configured for MINGW build
# Script is involutoric
#
# Author: Sam Ginrich
# No warranty of any kind!
#
#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
#
# Definition of Setup parameters
# these are entered into configuration files, if not already there, never modified!
#
#
#buildbits= # does nothing
#buildbits=32 # configure 32-bit build
#buildbits=64 # configure 64-bit build
buildbits=64
#winsock2=0 # configure parameter to exclude winsock2 library
#winsock2=1 # configure parameter to include winsock2 library
#winsock2= # does nothing, same effect as winsock2=1
winsock2=
# Issue with header "$ACE_ROOT/ace/OS_NS_stdlib.h"
# In some MINGW installation, the compiler is confused with a defined 'rand_r' macro
# This takes effect when building TAO, not ACE
#
#rand_r_issue= # does nothing, suggested initial value
#rand_r_issue=1 # modifies "$ACE_ROOT/ace/OS_NS_stdlib.h" to #undef-ine macro 'rand_r',
# before impact, suggested when issue occurs
rand_r_issue=
#
#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
echo "ACE/TAO Build Target Values"
echo ""
echo "buildbits=$buildbits"
echo "winsock2=$winsock2"
echo "rand_r_issue=$rand_r_issue"
echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
echo ""
echo "STEP: Enter ACE_Wrappers and define locations"
cd ./ACE_Wrappers
# Check, whether we arrives there ...
if [ ! -f "./ACE-INSTALL.html" ]; then
echo "ACE_Wrappers missing or invalid ... STOP"
exit 1
fi
export ACE_ROOT=${PWD}
export TAO_ROOT=${ACE_ROOT}/TAO
# Set C-Config Header for Windows
echo '#include "ace/config-win32.h"' > ace/config.h
# Set Platform MINGW
pl_macro=$ACE_ROOT/include/makeinclude/platform_macros.GNU
pl_mingw='$(ACE_ROOT)/include/makeinclude/platform_mingw32.GNU'
echo "include $pl_mingw" > $pl_macro
if [ "$buildbits" != "" ];
then
echo "------------------------------------------------------------"
echo ""
echo "STEP: Provide support for 64-bit build in 'platform_g++_common.GNU'"
pl_gpp=$ACE_ROOT/include/makeinclude/platform_g++_common.GNU
donetag2=" FLAGS_C_CC += -m64"
marker2="CCFLAGS += -Wnon-virtual-dtor"
buildbitsSwitch="ifeq (\$(buildbits),32)\n FLAGS_C_CC += -m32\n LDFLAGS += -m32\nendif\nifeq (\$(buildbits),64)\n FLAGS_C_CC += -m64\n LDFLAGS += -m64\nendif"
case `grep -Fx "$donetag2" "$pl_gpp" >/dev/null; echo $?` in
0)
echo "File $pl_gpp already modified "
;;
1)
# anyway store a copy
cp $pl_gpp /tmp
echo "copy of original $pl_gpp stored in \\tmp"
echo "insert compiler switches for buildbits rule"
sed -i "s/$marker2/$marker2\n\n$buildbitsSwitch/g" $pl_gpp
;;
*)
echo "Error scanning file $pl_gpp"
;;
esac
echo "------------------------------------------------------------"
echo ""
pl_mingw=$ACE_ROOT/include/makeinclude/platform_mingw32.GNU
echo "STEP: Set parameter for 64-bit build in $pl_mingw"
donetag3="buildbits =.*"
marker3="mingw32 = 1"
buildbitsDef="# 32\/64-bit build\n# parameter 'buildbits' is applied in platform_gnuwin32_common.GNU\nbuildbits = $buildbits"
case `grep -Ex "$donetag3" "$pl_mingw" >/dev/null; echo $?` in
0)
echo "File $pl_mingw already modified "
echo "Verify value! "
grep "buildbits =" $pl_mingw
;;
1)
# anyway store a copy
cp $pl_mingw /tmp
echo "copy of original $pl_mingw stored in \\tmp"
echo "insert buildbits=$buildbits"
sed -i "s/$marker3/$marker3\n\n$buildbitsDef/g" $pl_mingw
;;
*)
echo "Error scanning file $pl_mingw"
;;
esac
fi
if [ "$winsock2" != "" ];
then
echo "------------------------------------------------------------"
echo ""
#pl_mingw=$ACE_ROOT/include/makeinclude/platform_mingw32.GNU
echo "STEP: Winsock lack control"
donetag4="winsock2 =.*"
marker4=$marker3
winsockDef="winsock2 = $winsock2"
# $donetag4 is regular expression, -E
case `grep -Ex "$donetag4" "$pl_mingw" >/dev/null; echo $?` in
0)
echo "File $pl_mingw already modified "
echo Verify Value!
grep "winsock2 =" $pl_mingw
;;
1)
# anyway store a copy
cp $pl_mingw /tmp
echo "copy of original $pl_mingw stored in \\tmp"
echo insert $winsockDef
sed -i "s/$marker4/$marker4\n\n$winsockDef/g" $pl_mingw
;;
*)
echo "Error scanning file $pl_mingw"
;;
esac
fi
if [ "$rand_r_issue" == "1" ];
then
echo "------------------------------------------------------------"
echo ""
echo "STEP: Handle issue with defined C-macro rand_r"
onsll=$ACE_ROOT/ace/OS_NS_stdlib.h
donetag1="//#rand_undefined"
case `grep -Fx "$donetag1" "$onsll" >/dev/null; echo $?` in
0)
echo "File $onsll already modified"
;;
1)
# anyway store a copy
cp $onsll /tmp
echo "copy of original $pl_gpp stored in \\tmp"
echo "insert '#undef rand_r'"
sed -i 's/#if !defined (ACE_LACKS_RAND_R)/\/\/#rand_undefined\n#undef rand_r\n#if !defined (ACE_LACKS_RAND_R)/g' $onsll
;;
*)
echo "Error scanning file $onsll"
;;
esac
fi
echo "============================================================"
echo ""
echo "Content of "$ACE_ROOT/ace/config.h" is"
cat "ace/config.h"
echo ""
echo Content of "$pl_macro" is
cat $pl_macro
echo "-------------------------------------------------------------"
echo ""
echo ""
echo ""
echo "Suggested BUILD STEPS:"
echo ""
echo ""
echo "# 1. Define context"
echo "export ACE_ROOT=${PWD}"
echo "export TAO_ROOT=${ACE_ROOT}/TAO"
echo ""
echo "# 2. Build ACE"
echo 'cd ${ACE_ROOT}/ace'
echo "make -f GNUmakefile"
echo ""
echo "# 3. Verify ACE"
echo 'cd ${ACE_ROOT}/tests'
echo "make -f GNUmakefile"
echo "perl run_test.pl"
echo "#NOTE: Windows Firewall will ask for permission for each upcoming server instance"
echo ""
echo "# 4. Build TAO"
echo 'cd ${TAO_ROOT}'
echo "make -f GNUmakefile"
echo ""
echo "# 5. Basic TAO verification"
echo 'cd ${TAO_ROOT}/tests'
echo "make -f GNUmakefile"
echo 'cd ${TAO_ROOT}/tests/Param_Test'
echo "perl run_test.pl"
I'm running VisualSVN on a Windows server.
I'm trying to add a post-commit hook to update our staging project whenever a commit happens.
In VisualSVN, if I type the command in the hook/post-commit dialog, everything works great.
However, if I make a batch file with the exact same command, I get an error that says the post-commit hook has failed. There is no additional information.
My command uses absolute paths.
I've tried putting the batch file in the VisualSVN/bin directory, I get the same error there.
I've made sure VisualSVN has permissions for the directories where the batch file is.
The only thing I can think of is I'm not calling it correctly from VisualSVN. I'm just replacing the svn update command in the hook/post-commit dialog with the batch file name ("c:\VisualSVN\bin\my-batch-file.bat") I've tried it with and without the path (without the path it doesn't find the file at all).
Do I need to use a different syntax in the SVNCommit dialog to call the batch file? What about within the batch file (It just has my svn update command. It works if I run the batch file from the command line.)
Ultimately I want to use a batch file because I want to do a few more things after the commit.
When using VisualSVN > Select the Repo > Properties > Hooks > Post-commit hook.
Where is the code I use for Sending an Email then running a script, which has commands I want to customize
"%VISUALSVN_SERVER%\bin\VisualSVNServerHooks.exe" ^
commit-notification "%1" -r %2 ^
--from support#domainname.com --to "support#domainname.com" ^
--smtp-server mail.domainname.com ^
--no-diffs ^
--detailed-subject
--no-html
set PWSH=%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe
%PWSH% -command $input ^| C:\ServerScripts\SVNScripts\post-commit-wp.ps1 %1 %2
if errorlevel 1 exit %errorlevel%
The script file is located on C:\ServerScripts\SVNScripts\
post-commit-wp.ps1 and I pass in two VisualSVN variables as %1 and %2
%1 = serverpathwithrep
%2 = revision number
The script file is written in Windows PowerShell
# PATH TO SVN.EXE
$svn = "C:\Program Files\VisualSVN Server\bin\svn.exe"
$pathtowebistesWP = "c:\websites-wp\"
# STORE HOOK ARGUMENTS INTO FRIENDLY NAMES
$serverpathwithrep = $args[0]
$revision = $args[1]
# GET DIR NAME ONLY FROM REPO-PATH STRING
# EXAMPLE: C:\REPOSITORIES\DEVHOOKTEST
# RETURNS 'DEVHOOKTEST'
$dirname = ($serverpathwithrep -split '\\')[-1]
# Combine ServerPath with Dir name
$exportpath = -join($pathtowebistesWP, $dirname);
# BUILD URL TO REPOSITORY
$urepos = $serverpathwithrep -replace "\\", "/"
$url = "file:///$urepos/"
# --------------------------------
# SOME TESTING SCRIPTS
# --------------------------------
# STRING BUILDER PATH + DIRNAME
$name = -join($pathtowebistesWP, "testscript.txt");
# CREATE FILE ON SERVER
New-Item $name -ItemType file
# APPEND TEXT TO FILE
Add-Content $name $pathtowebistesWP
Add-Content $name $exportpath
# --------------------------------
# DO EXPORT REPOSITORY REVISION $REVISION TO THE ExportPath
&"$svn" export -r $revision --force "$url" $exportpath
I added comments to explain each line and what it does. In a nutshell, the scripts:
Gets all the parameters
Build a local dir path
Runs SVN export
Places files to a website/publish directory.
Its a simple way of Deploying your newly committed code to a website.
Did you try to execute batch file using 'call' command? I mean:
call C:\Script\myscript.bat
I was trying the same thing and found that you also must have the script in the hooks folder.. the bat file that is.