We have a server that we use to build our software and sometimes it is overloaded. I was told that our servers network configured in such a way that I could build/compile our source code on another machine that might be not so overloaded at that time via bsub.
So from the directory where my Makefile (/home/username/workspace) is, I enter:
$ bsub make clean
It seems that this command successfully send 'make clean' to another machine because a few minutes later I got notification via email. However, it seems that 'make clean' has failed. This is what the email says:
Job <make clean> was submitted from host <myserver> by user <username> in cluster <Cluster_Name>.
Job was executed on host(s) <hostname>, in queue <long>, as user <username> in cluster <Cluster_Name>.
...
...
</home/username> was used as the home directory.
</tmp> was used as the working directory.
...
...
make: *** No rule to make target `clean'. Stop.
So, it appears that /tmp was used as the working directory instead of my /home/username/workspace where the Makefile is.
Now the questions are:
How do I make my workspace as the working directory? OR
How do I copy all the necessary files to the /tmp? OR
Maybe I'm doing this all wrong?
I don't have any background at all on distributed computing or related fields, so I might not be asking the right questions.
Thanks in advance!
Related
I cant get codecollaborator to upload files to for code review. I suspect I am missing some config. I have been scouring perforce and smartbear and stackover flow pages for a couple hours now no luck
CENTOS7
p4 (cant seem to find the version)
Collaborator Enterprise v11.2.11200
My p4 works totally fine have been using for months now to create CLs and submit. But now i need to upload files for code reviews.
command i ran to setup ccollab:
wget https://s3.amazonaws.com/downloads.smartbear/collaborator/11.2.11200/ccollab_client_11_2_11200_unix.sh
chmod +x ccollab_client_11_2_11200_unix.sh
./ccollab_client_11_2_11200_unix.sh
(went through install accepting entering as prompted)
ccollab login https://<codecollaborator_server> <username>
the above logs in fine no errors
ccollab --no-browser --scm perforce --server-proxy-host https://codecollaborator_server --p4user <username> --p4charset utf8 --p4client local_workspace_name --p4 /bin/p4 set
the try to upload a file
ccollab --debug addchangelist new 123456789
and get the following output:
Connecting to server at https://
Connected to Collaborator Enterprise v11.2.11200
Connected as:
Attaching changelists to review
Auto-detecting SCM System for '/my/workspace/path'
Checking client configuration for '/my/workspace/path'.
ERROR: Could not configure SCM system:
SCM system could not be auto-detected, but there was an error: Cannot run program "accurev" (in directory "/my/workspace/path"): error=2, No such file or directory
I tried to find what the "accurev" package is or how to use it but no joy.
Accurev is a different source control system. Sounds like Code Collab doesn't know that it's supposed to be using Perforce?
I am migrating a Django application from Openshift v2 to v3 (In case you don't know, RedHat is shutting down v2 on September 30th, see: https://blog.openshift.com/migrate-to-v3-v2-eol/)
So, I am following this blog post to help me: https://blog.openshift.com/migrating-django-applications-openshift-3/ . I am new to all these Docker / Kubernetes concepts the new version is build upon.
I was able to make some progress : I managed to get a successful build of my app. Yet it crashes at deployment time:
---> Running application from script (app.sh) ...
/usr/libexec/s2i/run: line 42: /opt/app-root/src/app.sh: Permission denied
Indeed, app.sh has lost its x permission. I log into the failing container as debug and see it:
> oc debug dc/<my app>
> (app-root)sh-4.2$ ls -l /opt/app-root/src/app.sh
-rw-rw-r--. 1 default root 127 Sep 6 21:20 /opt/app-root/src/app.sh
The blog posts states "Ensure that the app.sh file is executable by running chmod +x app.sh.", which I did on my local repo. Whatever, I want to do it again directly in the pod, but it doesn't work:
(app-root)sh-4.2$ chmod +x /opt/app-root/src/app.sh
chmod: changing permissions of ‘/opt/app-root/src/app.sh’: Operation not permitted
So, how can I set the x permission to app.sh ? Thank you
Without looking into more details, any S2I builder image will gladly use your custom supplied run script to start the application in an alternative way.
Create .s2i/bin/ (mind the dot) in your source code directory, place the run script into it and rebuild the app in OpenShift - it will automatically use your custom run script upon deployment.
This is the preferred way of starting applications using custom commands in OpenShift.
Regarding your immediate problem, there is a very simple reason why you can not change the permissions of the script: you were trying to modify the permissions in the deployed pod, and not the builder pod. Deployed pods run using different UIDs, usually somewhere in the range of 100000000, and definitely do not match the file ownership as generated by the build. Hence permission denied.
The root cause of your problem (app.sh losing executable permissions) must be in the way the build process installs those files, and indeed looking at the /usr/libexec/s2i/assemble script in the base image does seem to reveal the culprit. The last two lines are:
# set permissions for any installed artifacts
fix-permissions /opt/app-root
If you wanted to change this part of the build instead of using a custom run script, I suggest you then create .s2i/bin/assemble in your project's source code and make it look sort of like this:
#!/bin/bash
echo "Running stock build:"
${STI_SCRIPTS_PATH}/assemble
echo "Fixing the mess:"
chmod 755 /opt/app-root/src/app.sh
This will fix whatever the stock build process does to file permissions, and will do it using the same UID as the rest of the build, so file ownership shouldn't be an issue.
as I stumbled upon this issue myself I've found a way to resolve it.
You have to make your file app.sh executable and push it in your repo as such.
If git does not track this modification as it did for me, you have to use: git update-index --chmod=+x app.sh for it to work.
This is Driving me nuts.
I had a working .ebextensions config file in my Project which was working fine.
Recently my single instance failed and a new one got initiated. My configuration failed to run so i tried to troubleshoot what went wrong. I didn't find anything suspicious so i just created a new .config with a very simple command but it still fails!!
I validated my config file with an online yaml validator.
I Connected to the instance through remote desktop and saw that .ebextensions folder is actually created within the wwwroot and then it disappears meaning that it got successfully picked up by elastic beanstalk.
I also granted all permissions to everyone on the test folder just to make sure this is not the reason.
Whichever i tried the old configuration or this test command it just does not work and elastic beanstalk just ignores it!
Any info of what might be wrong is appreciated.
commands:
01_Dowork:
command: mkdir kakarot
cwd: c:\\testdir
waitForCompletion: 0
I think everything under 01_DoWork needs to be indented (command, cwd, waitForCompletion). Also, make sure you're using spaces and not tabs.
Check the properties on your config file in VS. It should be (I think) both 'Content' and 'Copy if Newer'. Also, make sure that it gets packaged into the msdeploy package. It's a .zip file in/below your obj directory.
The command will error-out of it's already succeeded, so you would want to either ignore errors or add this. I found this syntax on another SO post but don't know who to credit for it :-/. The errorlevel will cause your command to not run if the directory already exists.
test: test ! -d c:\\testdir\\kakarot
If you're creating a package.zip (that inside has a deploy manifest json file plus the actual site.zip content) for a Windows deployment, it appears the .ebextensions directory needs to be inside package.zip, alongside the manifest json, not inside the site.zip, contrary to the current documentation.
I'm trying to come up with some sensible solution for a build written using SCons, which relies on quite a lot of applications to be accessible in a Unix-like way, using Unix-like paths etc. However, when I'm trying to use SCons plugin, or Git plugin in Jenkins, it tries to invoke the plugins using something like cmd /c git.exe - and this will certainly fail, because Git was installed using Cygwin and is only known in Cygwin shell, but not in CMD. But even if I could make git and the rest available to cmd.exe, other problems arise: the Cygwin version of Git expects paths to have forward slashes and treats backward slashes as escape characters. Idiotic Windows file-system related issues kick in too (I can't give Jenkins permissions to delete my own files!).
So, is there a way to somehow make Jenkins only use Cygwin shell, and never cmd.exe? Or should I be prepared to run some Linux in a VM to have this handled?
You could configure Jenkins to execute the cygwin command with the specific shell command, as follows:
c:\cygwin\bin\mintty --hold always --exec /cygdrive/c/path/to/bash/script.sh
Where script.sh will execute all the commands needed for the Jenkins execution.
Just for the record here's what I ended up doing:
Added a user SYSTEM to Cygwin, mkpasswd -u SYSTEM
Edited /etc/passwd by adding the newly created user's home directory to the record. Now it looks something like the below:
SYSTEM:*:18:544:,S-1-5-18:/home/SYSTEM:
Copied my own user's configuration settings such as .netrc, .ssh and so on into the SYSTEM home. Then, from Windows Explorer, through an array of popups I've claimed ownership of all of these files to SYSTEM user. One by one! I love Microsoft!
In Jenkins I now run a wrapper for my build that sets some other environment variables etc. by calling c:\cygwin\bin\bash --login -i /path/to/script/script
Gave it up because of other difficulties in configuration and made Jenkins service run under my user rather then SYSTEM. Here's a blog post on how to do it: http://antagonisticpleiotropy.blogspot.co.il/2012/08/running-jenkins-in-windows-with-regular.html but, basically, you need to open Windows services, then find Jenkins service, select it's properties, go to "Logon" tab and change the user to the "this user".
One way to do this is to start your "execute shell" build steps with
#!c:\cygwin\bin\bash --login
The trick is of course that it resets your current directory so you need to
cd `cygpath $WORKSPACE`
to get back to the workspace.
Adding to thon56's good answer: this is helpful: "set -ex"
#!c:\cygwin\bin\bash --login
cd `cygpath $WORKSPACE`
set -ex
Details:
-e to exit on error. This is important if you want your jobs to fail on error.
-x to echo command to the screen, if desired.
You can also use #!c:\cygwin\bin\bash --login -ex, but that echos a lot of login steps that you most likely don't care to see.
When I build project from terminal by using 'xcodebuild' command I succeed, but when I try to do run same script from cron task I receive error
"Code Sign error: The identity '****' doesn't match any valid certificate/private key pair in the default keychain"
I think problem is in settings and permissions of crontab utility, it seems crontab does not see my keychain
Can anyone provide me terminal command how to make my keychain visible for crontab
I encountered a similar issue with trying to build nightly via cron. The only resolution I found was to create a plist in /Library/LaunchDaemons/ and load it via launchctl. The key necessary is "SessionCreate" otherwise you will quickly run in to problems similar to what was encountered with trying to use cron -- namely that your user login.keychain is not available to the process. "SessionCreate" is similar to "su -l" in that (as far as I understand) it simulates a login and thus default keychains you expect will be available; otherwise, you are stuck with only the System keychain despite the task running as your user.
I found the answers (though not the top answer currently) here useful in troublw shooting this issue: Missing certificates and keys in the keychain while using Jenkins/Hudson as Continuous Integration for iOS and Mac development
You execute your cron job with which account ?
most probably the problem !!
You can add
echo `whoami`
at the beginning of your script to see with which user the script is launched.
Also when a Bash script is launched from cron, it don't use the same environment variable (non login shell) as when you launch it as a user.
When the script launches from cron, it doesn't load your $HOME/.profile (or .bash_profile). Anything you run from cron has to be 100% self-sufficient in terms of it's environment. I'd suggest you make yourself a file called something like "set_build_env.sh" It should contain everything from your .profile that you need to build, such as $PATH, $HOME, $CLASSPATH etc. Then in your build script, load set_build_env.sh using the dot notation or source cmd as ericc said. You should also remove the build-specific lines from your.profile and then source set_build_env from there too so only one place to maintain. Example:
source /home/dmitry/set_build_env.sh #absolute path
. /home/dmitry/set_build_env.sh #dot-space notation same as "source"