We are running a shell script that runs a deployed IDQ mapping. I tried in unix directories to see if it created a mapping log file but no where i can see.
I checked in various directories under "" <infa_home> " folder but i could not trace the log file.
If you have come across the same situation, please do let me know.
Which log you are looking for? Mapping/ workflow/ DIS/Webservice/Catalina?
All IDQ logs stored under $HOME/infa_shared/log/disLogs/*
Related
I'm using the Hadoop library to upload files in S3. Because of some metric configuration file is missing I'm getting this exception
MetricsConfig - Could not locate file hadoop-metrics2-s3a-file-system.properties org.apache.commons.configuration2.ex.ConfigurationException:
Could not locate: org.apache.commons.configuration2.io.FileLocator#77f46cee[fileName=hadoop-metrics2-s3a-file-system.properties,basePath=<null>,sourceURL=,encoding=<null>,fileSystem=<null>,locationStrategy=<null>]
My current configurations are
configuration.set("fs.s3a.access.key", "accessKey")
configuration.set("fs.s3a.secret.key", "secretKey")
Where to add this configuration file? What to add to that configuration file?
don't worry about it, it's just an irritating warning. It's only relevant when you have the s3a or abfs connectors running in a long-lived app where the metrics are being collected and fed to some management tooling.
Set the log level to warn in the log4j.properties file in your spark conf dir
log4j.logger.org.apache.hadoop.metrics2=WARN
I just placed an empty file in the classpath and it stopped complaining. Like:
touch /opt/spark/conf/hadoop-metrics2-s3a-file-system.properties
I am using the Visual Studio AWS add-on/plugin to deploy my application, but want to move to a CI/CD server and scripted deployment.
I've installed the AWS SDK for Windows and thus want to use the awsdeploy.exe command line to accomplish this.
I've used msbuild and a publish profile to create the .zip deployable of my application (ASP.NET WebApi project)
I've put together the following command line command:
awsdeploy.exe -r -w -v -l "C:\<path_to>\deploylog.txt" "-DDeploymentPackage=C:\<path_to>\my_app.zip" "-DAWSAccessKey=<my_access_key>" "-DAWSSecretKey=<my_secret_key>" "C:\<path_do>\AWSDeployConfiguration.txt"
The "AWSDeployConfiguration.txt" file is what was generated by VisualStudio when I did the first deployment.
RESULT:
The console output and the text written to the log is:
INFO - Scanning configuration.
INFO - ...inspecting application '<my_app_name>' for environment '<my_environment_name>' and version 'v20180918223701'
Nothing happens with the ELB application.
What am I missing and/or how do I get more information to figure this out?
I posted this question on the AWS forums and got the following answer that also worked for me.
Hi! I have this same what You when I trying run this from cmd. But it You will try check what application is returning You will see that value is 3. Generally everything !=0 is error.
What I did?
1. I checked with Process Monitor if application is doing any network request to AWS - no it even not trying. https://learn.microsoft.com/en-us/sysinternals/downloads/procmon
I decided to recompile awasdeploy.exe and I found out that in the main procedure is a try... catch.. without any logs and just return(3). I added some logs and get a detailed error - look at attached image.
After few attempts I get a list of missing dll files:
AWSSDK.MobileAnalytics.dll
AWSSDK.CognitoIdentity.dll
All these files I found in: C:\Program Files (x86)\AWS SDK for .NET\bin and just simply copied to: C:\Program Files (x86)\AWS Tools\Deployment Tool (next to awsdeploy.exe)
Now deploy is working again.
Hi Jenkins and AWS Guru's
I already look online for any possible solutions but not getting a solution for my problem. I just issued an "eb --version" on Jenkins execute shell under a test project but getting "eb: command not found" during the execution.
Wierd thing is if I issue the same command on the Jenkins box via CLI I'm getting a good response from it. Any suggestions for the fix please? thanks in advance
Your Jenkins setup has a different path than the user you logged in with.
There are two solutions:
Add the path to the executable in the PATH environment variable. Use where eb to find the correct path. Then in Jenkins, click on
Manage Jenkins -> Configure System, Global Properties. Check Environment Variables. Set Name to PATH. Set Value to $PATH:/path/to/eb. Then restart Jenkins.
Call the eb command with its fully qualified path.
EDIT: Added steps to update path in Jenkins.
this is now fixed, need to create a properties file that was basically a copy of /var/lib/jenkins/.bash_profile file which would have the correct paths and add that in Jenkins settings. Allowing it to get the required paths. After properties file is created you need to set it on Jenkins-Configure section, Place a check on Prepare jobs environment then set the full path of the properties file (/var/lib/jenkins/environment_variables.properties) on the Properties File Path and restart Jenkins
This is Driving me nuts.
I had a working .ebextensions config file in my Project which was working fine.
Recently my single instance failed and a new one got initiated. My configuration failed to run so i tried to troubleshoot what went wrong. I didn't find anything suspicious so i just created a new .config with a very simple command but it still fails!!
I validated my config file with an online yaml validator.
I Connected to the instance through remote desktop and saw that .ebextensions folder is actually created within the wwwroot and then it disappears meaning that it got successfully picked up by elastic beanstalk.
I also granted all permissions to everyone on the test folder just to make sure this is not the reason.
Whichever i tried the old configuration or this test command it just does not work and elastic beanstalk just ignores it!
Any info of what might be wrong is appreciated.
commands:
01_Dowork:
command: mkdir kakarot
cwd: c:\\testdir
waitForCompletion: 0
I think everything under 01_DoWork needs to be indented (command, cwd, waitForCompletion). Also, make sure you're using spaces and not tabs.
Check the properties on your config file in VS. It should be (I think) both 'Content' and 'Copy if Newer'. Also, make sure that it gets packaged into the msdeploy package. It's a .zip file in/below your obj directory.
The command will error-out of it's already succeeded, so you would want to either ignore errors or add this. I found this syntax on another SO post but don't know who to credit for it :-/. The errorlevel will cause your command to not run if the directory already exists.
test: test ! -d c:\\testdir\\kakarot
If you're creating a package.zip (that inside has a deploy manifest json file plus the actual site.zip content) for a Windows deployment, it appears the .ebextensions directory needs to be inside package.zip, alongside the manifest json, not inside the site.zip, contrary to the current documentation.
When I build project from terminal by using 'xcodebuild' command I succeed, but when I try to do run same script from cron task I receive error
"Code Sign error: The identity '****' doesn't match any valid certificate/private key pair in the default keychain"
I think problem is in settings and permissions of crontab utility, it seems crontab does not see my keychain
Can anyone provide me terminal command how to make my keychain visible for crontab
I encountered a similar issue with trying to build nightly via cron. The only resolution I found was to create a plist in /Library/LaunchDaemons/ and load it via launchctl. The key necessary is "SessionCreate" otherwise you will quickly run in to problems similar to what was encountered with trying to use cron -- namely that your user login.keychain is not available to the process. "SessionCreate" is similar to "su -l" in that (as far as I understand) it simulates a login and thus default keychains you expect will be available; otherwise, you are stuck with only the System keychain despite the task running as your user.
I found the answers (though not the top answer currently) here useful in troublw shooting this issue: Missing certificates and keys in the keychain while using Jenkins/Hudson as Continuous Integration for iOS and Mac development
You execute your cron job with which account ?
most probably the problem !!
You can add
echo `whoami`
at the beginning of your script to see with which user the script is launched.
Also when a Bash script is launched from cron, it don't use the same environment variable (non login shell) as when you launch it as a user.
When the script launches from cron, it doesn't load your $HOME/.profile (or .bash_profile). Anything you run from cron has to be 100% self-sufficient in terms of it's environment. I'd suggest you make yourself a file called something like "set_build_env.sh" It should contain everything from your .profile that you need to build, such as $PATH, $HOME, $CLASSPATH etc. Then in your build script, load set_build_env.sh using the dot notation or source cmd as ericc said. You should also remove the build-specific lines from your.profile and then source set_build_env from there too so only one place to maintain. Example:
source /home/dmitry/set_build_env.sh #absolute path
. /home/dmitry/set_build_env.sh #dot-space notation same as "source"