I'm using the ext-email extension to extract the User Story which is added as a commit message to include in the mail body.
This is the console output:
Commit message: "US285568"
I used the Build Log Excerpt method of the ext-email plug-in as follows:
STORY: ${BUILD_LOG_EXCERPT, start="Commit message:\ \"", end="\'"}
However, this does not match anything and I'm not able to understand why it's failing.
I couldn't find a proper documentation for this plug-in.
I used a work-around solution by triggering a helper job at the end of the current job (which contains the commit message in the console output).
I am executing the following shell code in the helper job:
result=$(curl -GET {JENKINS_IP}/jenkins/job/{UPSTREAM_JOB_NAME}/consoleFull --user "user:pass")
comm=$(grep "Commit message:" <<< "$result")
if grep -E "US[0-9]+" <<< "$comm"
then
final=$(grep -o "US[0-9]*" <<< "$comm")
else
final="<font color=\"red\">User Story not found</font>"
fi
echo FINAL=$final > env.properties
To access the FINAL variable (which contains the value of the user story) as an environment variable, I have used the EnvInject plug-in.
To access the FINAL variable outside the shell code in the job, add the "Inject environment variables" build step after the shell code and enter "env.properties" in the Properties File Path.
Related
Summary of the problem
I've setup a repository with the checks I developed, designed to be used exclusively in an another repository.
The problem I've been facing is that when I run pre-commit run -a -v the checks don't go through every file, and time to time they even change!
A little more details
I've run the [identity][1] check and it prints every file in the repo, meaning the files are read by pre-commit (per my understanding)
When I execute pre-commit run the file in staging are correctly parsed
Why do i think that the checks don't run on every file?
I always print something to the standard output for every check, and when I run it verbosely it only prints 5 to 7 lines, meaning it checked only those.
If I try and edit a file (breaking a check) whose not on the short list, the check still passes through
Some code
The structure of the folder containing the files to be checked
./
articles/
some_name/
file.md
file.png
and_so_on.jpeg
team/
some_other_name/
file.md
propic.jpg
Summary of the .pre-commit-hooks.yaml
- id: team-check-name
name: blabla
description: blabla
entry: team-checkname
files: 'team/.*/'
types: [markdown]
language: python
- id: article-check-name
name: blabla
description: blabla
entry: article-checkname
files: 'articles/.*/'
types: [markdown]
language: python
Essentially the checks supposed to run on articles/.*/ start with article- and they all have this property: files: 'articles/.*/'.
Similarly every check supposed to run on team/.*/ starts with team- and they all have files: 'team/.*/'
Summary of the .pre-commit-config.yaml
repos:
- repo: https://github.com/repourl
rev: commit_hash
hooks:
- id: team-check-name
- id: article-check-name
But as you can see, I don't override any settings so it shouldn't interfere
the hook tools you've written are incorrect -- they only process sys.argv[1] rather than positional arguments
your code uses a lot of global variables so it's not a straightforward refactor -- typically you'd either use argparse to collect nargs='*' or loop over sys.argv[1:] (if you don't have any options)
as to why this is the convention -- it's very wasteful to start a linter process over and over to lint a single file (often times executable startup cost dwarfs the actual linting / formatting process)
disclaimer: I wrote pre-commit
I have simple console application that prints environment variables:
int main(int argc, char **argv, char **envp)
{
printf("Scanning for LD_LIB_PATH\n");
for (char **env = envp; *env != 0; env++)
{
char *thisEnv = *env;
std::string s= *env;
if (s.find("LD_LIBRARY_PATH")!=std::string::npos)
{
printf("!!!!!!! %s\n", thisEnv);
}
printf("%s\n", thisEnv);
}
}
Before run executable I run script that sets LD_LIBRARY_PATH
export LD_LIBRARY_PATH=~/aaa/bbb/Debug:~/ccc/ddd/Debug
echo "searching:"
export | grep LD_LIBRARY
echo "done"
Script run fine with output:
exporting
searching:
declare -x LD_LIBRARY_PATH="/home/vicp/aaa/bbb/Debug:/home/vico/ccc/ddd/Debug"
done
I run executable and it finds many variables but no environment variable LD_LIB_PATH. Why?
UPD
As recommended I script . ./script.sh
Then double check with command:
export |grep LD_LIBRARY_PATH
Got output:
declare -x LD_LIBRARY_PATH="/home/vicp/aaa/bbb/Debug:/home/vico/ccc/ddd/Debug"
But still don't see LD_LIBRARY_PATH in my programm.
Depending on how you run the script, the env variable will only be added to the environment of the subshell running the script.
If you run the script like this:
$ ./script.sh
This will spawn a new shell wherein the script is run. The parent shell, i.e. the one you started the script from, will not be affected by what the script does. Changes to the environment will therefore not be visible (changing the working directory or similar will also not work).
If the script is intended to modify the environment of the current shell, the script needs to be sourced using the . or source commands:
$ . ./script.sh
$ source ./script.sh
From help source in bash:
Execute commands from a file in the current shell.
Then there seems to be a problem with the code:
As a commenter stated previousy, there are a lot of exclamation points in the success case printf. I presume these were meant to highlight the found variable name. Since they are outside of the opening quote of the format string, they are interpreted as logical not operators.
The result is that the format string literal (a pointer) is negated, resulting in false due to the number of operators. This usually maps to 0, which would mean the format string becomes a null pointer. What happens when handing a null pointer to printf as the format string depends, as far as I can tell, on the implementation. If it doesn't crash, it most certainly will not print anything however.
So the code may actually work, but there is no visible output due to that bug. To fix it, move the exclamation points into the format string, i.e. after the opening quote.
Look at the line printf(!!!!!!!"%s\n", thisEnv);
Change to printf("!!!! %s\n", thisEnv);
It has nothing to do with your C/C++ application.
try the following:
$ ./script.sh
$ echo $LD_LIBRARY_PATH
And you'll see that LD_LIBRARY_PATH is not set
When you launch your script, bash creates a new process with its environment inherited from the original bash process. In that newly created process, you set the process environment variable LD_LIBRARY_PATH=xxxx
When finalized your script.sh exits, and its environment dies with it.
Meaning the LD_LIBRARY_PATH is not set in your original shell.
As mentioned here above you need to run your script in the current shell
Either with . or with source.
I tested with your C/C++ and it works
I have a program that needs to run a program we'll call externalProg in parallel on our linux (CentOS) cluster - or rather, it needs to run many instances of externalProg, each on different cores. Each "thread" creates 3 files based on a few parameters - the inputs to externalProg, a command file to tell externalProg how to execute my file, and a bash script to set up the environment (calls a setup script provided by the manufacturer) and actually call externalProg with my inputs.
Since this needs to be parallel with an unknown number of concurrent threads and I don't want to risk overwriting another thread's files, I am creating temp files using
mkstemp("PREFIX_XXXXXX")
for these input files. After the external program runs, I extract the relevant data and store it, and close the temp files (therefore deleting them).
We'll call the files created (Which actually have a name based on the template above)
tmpInputs - Inputs to externalProg
tmpCommand - Input that tells externalProg how to execute tmpInputs
tmpBash - bash script to set up and call externalProg with my inputs
The file tmpBash looks something like
source /path/to/setup/script # Sets up environment variables
externalProg < /path/to/tmpCommand
where tmpCommand is just a simple text file.
The problem I'm having is actually executing the bash script. Within my program, I call
ostringstream launchcmd;
launchcmd << "bash " << path_to_tmpBash
system(launchcmd.str().c_str());
But nothing happens. No error, no warning, no 'file not found' or permission denied or anything. I have verified that the files are being created and have the correct content. The rest of the code after system() is executed successfully (Though it fails since externalProg wasn't run).
Strangely, if I go back to the terminal and type
bash /path/to/tmpBash
then externalProg is executed successfully. I have also cout'd the launchcmd string, copy and pasted that in to the terminal, which also works successfully. For some reason, this only fails when called within my program.
After a bit of experimentation, I've determined that system() calls /bin/sh on our cluster. If I change launchcmd to look like
/path/to/tmpBash
(So that the full command should look like /bin/sh /path/to/tmpBash), I get a permission denied error, which is no surprise. The problem is that I can't chmod +x the tmpBash file while it's still open, and if I close the file, it gets deleted - so I'm not sure how to address that.
Is there something obviously wrong I'm doing, or does system() have some nuance that I'm missing?
edit: I wanted to add that I can successfully call things like
system("echo $PATH")
and get the expected results (in this case, my default $PATH).
Two separate ideas:
Change your SHELL environment variable to be /bin/bash, then call system(),
or:
Use execve directly `execve('/bin/bash', ['/path/to/tmpBash'], environ)
The goal of this code is to create a stack trace whenever a sigterm/sigint/sigsegv/etc is caught. In order to not rely on memory management inside of the C++ code in the case of a sigsegv, I have decided to write a bash script that will receive the PID and memory addresses in the trace array.
The Sig events are being caught.
Below is where I generate the call to the bash script
trace_size = backtrace(trace, 16);
trace[1] = (void *)ctx->rsi;
messages = backtrace_symbols(trace, trace_size);
char syscom[356] = {0};
sprintf(syscom,"bash_crash_supp.sh %d", getpid());
for (i=1; i<(trace_size-1) && i < 10; ++i)
{
sprintf(syscom,"%s %p",syscom,trace[i]);
}
Below is where my issue arises. The command in syscom is generating correctly. I can stop the code before the following popen, run the command in a terminal, and it functions correctly.
However running the script directly from the c++ code does not seem to work.
setuid(0);
FILE *bashCommand = popen(syscom,"r");
char buf[256] = {0};
while(fgets(buf,sizeof(buf),bashCommand) != 0) {
LogMessage(LOG_WARNING, "%s\n", buf);
}
pclose(bashCommand);
exit(sig);
The purpose of the bash script is to get the offset from /proc/pid/maps, and then use that to run addr2line to get the file name/line number.
strResult=$(sudo cat /proc/"$1"/maps | grep "target_file" | grep -m 1 '[0-9a-fA-F]')
offset=$( cut -d '-' -f 1 <<< "$strResult");
However offset is getting 0 when I run from the c++ code, but when I run the exact same command (that is stored in syscom in the c++ code) in a terminal, I get the expected output.
I have been trying to fix this for a while. Permissions are most likely the issue, but I've tried to work around these with every way I know of/have seen via google. The user trying to run the script (currently running the c++ code) is apache.
The fix does not need to worry about the security of the box. If something as simple as "chmod 777 /proc -r" worked, that would have been the solution (sadly the OS doesn't let me mess do such commands with /proc).
Things I've already tried:
Adding `` around the command that's stored in syscom from the c++ code
chown the script to apache
chmod 4755 on the bash_crash_supp.sh script, allowing it to always fire as root.
I have added apache to sudoers (visudo), allowing them to run sudo without using a password
I have added a sub file to sudoers.d (just in case) that does the same as above
I have looked into objdump, however it does not give me either the offset or the file/line num for an addr (from what I can see)
I have setuid(0) in the c++ code to set the current user to root
Command generated in C++
bash_crash_supp.sh 25817 0x7f4bfe600ec8 0x7f4bf28f7400 0x7f4bf28f83c6 0x7f4bf2904f02 0x7f4bfdf0fbb0 0x7f4bfdf1346e 0x7f4bfdf1eb30 0x7f4bfdf1b9a8 0x7f4bfdf176b8
Params in bash:
25817 0x7f4bfe600ec8 0x7f4bf28f7400 0x7f4bf28f83c6 0x7f4bf2904f02 0x7f4bfdf0fbb0 0x7f4bfdf1346e 0x7f4bfdf1eb30 0x7f4bfdf1b9a8 0x7f4bfdf176b8
Can anyone think of any other ways to solve this?
Long story short, almost all Unix-based systems ignore setuid on any interpreted script (anything with a shebang #!) as a security precaution.
You may use setuid on actual executables, but not the shell scripts themselves. If you're willing to take a massive security risk, you can make a wrapper executable to run the shell script and give the executable setuid.
For more information, see this question on the Unix StackExchange: https://unix.stackexchange.com/a/2910
I have this following simple program:
int main()
{
char* v = getenv("TEST_VAR");
cout << "v = " << (v==NULL ? "NULL" : v) << endl;
return 0;
}
These lines are added to .bashrc file:
TEST_VAR="2"
export TEST_VAR
Now, when I run this program from the terminal window (Ubuntu 10.04), it prints v = 2. If I run the program by another way: using launcher or from Eclipse, it prints NULL. I think this is because TEST_VAR is defined only inside bash shell. How can I create persistent Linux environment variable, which is accessible in any case?
On my system (Fedora 13) you can make system wide environment variables by adding them under /etc/profile.d/.
So for example if you add this to a file in /etc/profile.d/my_system_wide.sh
SYSTEM_WIDE="system wide"
export SYSTEM_WIDE
and then open a another terminal it should source it regardless of who the user is opening the terminal
echo $SYSTEM_WIDE
system_wide
Add that to .bash_profile (found in your home directory). You will need to log out and log back in for it to take effect.
Also, since you are using bash, you can combine the export and set in a single statement:
export TEST_VAR="2"
Sorry if I'm being naive but isn't .bash_profile useful only if you are running bash as your default shell ?
I 'sometimes' use Linux and mostly use ksh. I have .profile so may be you should check for .*profile and export the variable there.
Good luck :)
There is no such thing as a system-wide environment variable on Linux. Every process has its own environment. Now by default, every process inherits its environment from its parent, so you can get something like a system-wide environment by ensuring that a var is set in an ancestor of every process of interest. Then, as long as no other process changes that var, every process of interest will have it set.
The other answers here give various methods of setting variables early. For example, .bash_profile sets it in every login process a user runs, which is the ultimate parent of every process they run after login.
/etc/profile is read by every bash login by every user.