I'm writing a job-dsl seed job. The seed job needs to be able to generate from either github.com or from my companies github enterprise server. I'd like to keep one job rather than have two.
In each case I would like jenkins to be Authenticated. So to do this, I hardcoded the creds into the script. However I'm not satisfied with this. I would prefer to add a Credentials parameter on the seed job.
The problem is, the Creds parameter seems to add en ENV variable to the script that contains USERID/PASSWORD. http://steve-jansen.github.io/blog/2014/12/16/parsing-jenkins-secrets-in-a-shell-script/
However, the git jobdsl seems to want a Credentials ID, not USERID/PASSWORD.
How to resolve this impasse?
scm {
git {
remote {
name('origin')
url(repo)
credentials(myCredential)
}
branch('master')
}
}
A good introduction to the way Job DSL handles credentials, can be found on official wiki page.
Two examples explaining how it's possible to pass user password to Jenkins job:
// use the github-ci-key credentials for authentication with GitHub
job('example-1') {
scm {
git {
remote {
github('account/repo', 'ssh')
credentials('github-ci-key')
}
}
}
}
// assign the jarsign-keystore credentials to the PASSWORD build variable job('example-2') {
wrappers {
credentialsBinding {
usernamePassword('PASSWORD', 'jarsign-keystore')
}
}
}
Adding to the answer made by Duane above, CredentialID is the unique ID of a credential stored in Jenkins. You can also provide your own ID to identify it in a meaningful way.
Jenkins Credentials
Use this ID shown in the image above in your job-dsl script and it should work for you.
Peace.
Turns out that the premise of the question is false. CredentialsPlugin 1.7 sets the variable to the credentialID.
Related
Going through Okta's github repo it doesn't seem like there is an easy way to switch between AWS roles using their command line utility.
Normally I get temporary credentials using okta-aws default sts get-caller-identity. If I'm switching between projects and need to likewise switch my temporary credentials to a different role, the best method I've found thus far is to delete two files, ~/.okta/profile & ~/.okta/.current-session, then re-run the above command.
I've also found that when OKTA_STS_DURATION in .okta/config.properties is set for longer than is configured for that AWS role, you either need to wait until the local duration setting expires or reset the credentials process using the same method as above.
Is there a way to switch between or reset role based credentials using okta-aws command line utility?
Note: I'm using role as interchangeable with profile -- please correct me if this is inaccurate.
At this point the best solution is to switch over to gimme-aws-creds. After you run through the setup it ends up being easier to use, and appears to be better maintained by Nike.
Currently I am using Terraform and Aws Secrets Manager to store and retrieve secrets, and I would like to have some insight if my implementation is secure, and if not how can I make it more secure. Let me illustrate with what I have tried.
In secrets.tf I create a secret like (this needs to be implemented with targeting):
resource "aws_secretsmanager_secret" "secrets_of_life" {
name = "top-secret"
}
I then go to the console and manually set the secret in AWS Secrets manager.
I then retrieve the secrets in secrets.tf like:
data "aws_secretsmanager_secret_version" "secrets_of_life_version" {
secret_id = aws_secretsmanager_secret.secrets_of_life.id
}
locals {
creds = jsondecode(data.aws_secretsmanager_secret_version.secrets_of_life.secret_string)
}
And then I proceed to use the secret (export them as K8s secrets for example) like:
resource "kubernetes_secret" "secret_credentials" {
metadata {
name = "kubernetes_secret"
namespace = kubernetes_namespace.some_namespace.id
}
data = {
top_secret = local.creds["SECRET_OF_LIFE"]
}
type = "kubernetes.io/generic"
}
It's worth mentioning that I store tf state remotely. Is my implementation secure? If not, how can I make it more secure?
yes I can confirm it is secure since you accomplished the following:
plain text secrets out of your code.
Your secrets are stored in a dedicated secret store that enforces encryption and strict access control.
Everything is defined in the code itself. There are no extra manual steps or wrapper scripts required.
Secret manager support rotating secrets, which is useful in case a secret got compromised.
The only thing I can wonder about is using a Terraform backend that supports encryption like s3, and avoid commet the state file to your source control.
Looks good, as #asri suggests it a good secure implementation.
The risk of exposure will be in the remote state. It is possible that the secret will be stored there in plain text. Assuming you are using S3, make sure that the bucket is encrypted. If you share tf state access with other developers, they may have access to those values in the remote state file.
From https://blog.gruntwork.io/a-comprehensive-guide-to-managing-secrets-in-your-terraform-code-1d586955ace1
These secrets will still end up in terraform.tfstate in plain text! This has been an open issue for more than 6 years now, with no clear plans for a first-class solution. There are some workarounds out there that can scrub secrets from your state files, but these are brittle and likely to break with each new Terraform release, so I don’t recommend them.
Hi I'm working on similar things, here're some thoughts:
when running Terraform for the second time, the secret will be in plain text in state files which are stored in S3, is S3 safe enough to store those sensitive strings?
My work is using the similar approach: run terraform create an empty secret / dummy strings as placeholder -> manually update to real credentials -> run Terraform again to tell the resource to use the updated credentials. The thing is that when we deploy in production, we want the process as automate as possible, this approach is not ideal ut I haven't figure out a better way.
If anyone has better ideas please feel free to leave a comment below.
I am using Google Speech API in my Django web-app. I have set up a service account for it and am able to make API calls locally. I have pointed the local GOOGLE_APPLICATION_CREDENTIALS environment variable to the service account's json file which contains all the credentials.
This is the snapshot of my Service Account's json file:
I have tried setting heroku's GOOGLE_APPLICATION_CREDENTIALS environment variable by running
$ heroku config:set GOOGLE_APPLICATION_CREDENTIALS="$(< myProjCreds.json)"
$ heroku config
GOOGLE_APPLICATION_CREDENTIALS: {
^^ It gets terminated at the first occurrence of " in the json file which is immediately after {
and
$ heroku config:set GOOGLE_APPLICATION_CREDENTIALS='$(< myProjCreds.json)'
$ heroku config
GOOGLE_APPLICATION_CREDENTIALS: $(< myProjCreds.json)
^^ The command gets saved into the environment variable
I tried setting heroku's GOOGLE_APPLICATION_CREDENTIALS env variable to the content of service account's json file but it didn't work (because apparently the this variable's value needs to be an absolute path to the json file) . I found a method which authorizes a developer account without loading json accout rather using GOOGLE_ACCOUNT_TYPE, GOOGLE_CLIENT_EMAIL and GOOGLE_PRIVATE_KEY. Here is the GitHub discussion page for it.
I want something similar (or something different) for my Django web-app and I want to avoid uploading the json file to my Django web-app's directory (if possible) for security reasons.
Depending on which library you are using for communicating with Speach API you may use several approaches:
You may serialize your JSON data using base64 or something similar and set resulting string as one environment variable. Than during you app boot you may decode this data and configure your client library appropriately.
You may set each pair from credentials file as separate env variables and use them accordingly. Maybe library that you're using support authentication using GOOGLE_ACCOUNT_TYPE, GOOGLE_CLIENT_EMAIL and GOOGLE_PRIVATE_KEY similar to the ruby client that you're linking to.
EDIT:
Assuming that you are using google official client library, you have several options for authenticating your requests, including that you are using (service account): https://googlecloudplatform.github.io/google-cloud-python/latest/core/auth.html You may save your credentials to the temp file and pass it's path to the Client object https://google-auth.readthedocs.io/en/latest/user-guide.html#service-account-private-key-files (but it seems to me that this is very hacky workaround). There is a couple of other auth options that you may use.
EDIT2:
I've found one more link with the more robust approach http://codrspace.com/gargath/using-google-auth-apis-on-heroku/. There is ruby code, but you may do something similar in Python for sure.
Let's say the filename is key.json
First, copy the content of the key.json file and add it to the environment variable, let's say KEY_DATA.
Solution 1:
If my command to start the server is node app.js, I'll do echo $KEY_DATA > key.json && node app.js
This will create a key.json file with the data from KEY_DATA and then start the server.
Solution 2:
Save the data from KEY_DATA env variable in the some variable and then parse it to JSON, so you have the object which you can pass for authentication purposes.
Example in Node.js:
const data = process.env.KEY_DATA;
const dataObj = JSON.parse(data);
For the Jenkins Job DSL, I am trying to choose specific ssh agent (plugin) keys for a job (using the sshAgent keyword inside the wrappers context). We have the Jenkins ssh agent plugin installed and several keys setup (this plugin works, as we use it for almost all of our jobs). The Jenkins Job DSL sshAgent command always picks the first key, regardless of whether I specify a different key in our Jenkins setup.
I have tried using just the key name, but also tried key_name + space + description (just like the dropdowns show). That does not work either -- still picks the first key.
Is this a known issue? (I haven't turned up any searches for this yet)
You need to pass the ID of the credentials to the sshAgent DSL method. To get the ID, install at least version 1.21 of the Credentials Plugin. Then navigate to the credentials you want to use, e.g. if the credentials you want to use are global and called "Your Credentials" go to Jenkins > Credentials > Global credentials (unrestricted) > Your Credentials > Update. Then click the "Advanced..." button to reveal the ID. If you did not specify a custom ID when creating the credentials, it's a UUID like 99add9e9-84d4-408a-b644-9162a93ee3e4. Then use this value in your DSL script.
job('example') {
wrappers {
sshAgent('99add9e9-84d4-408a-b644-9162a93ee3e4')
}
}
It's recommended to use a recognizable custom ID when creating new credentials, e.g. deployment-key. That will lead to readable DSL scripts.
job('example') {
wrappers {
sshAgent('deployment-key')
}
}
I am passing my 3rd party subscription license key (newrelic) and few other config variables via the stack settings Custom JSON.
"opsworks_java": {
"datasources": {
"app": "jdbc/myapp"
}
},
"newrelic": {
"license": "2454645aef2e055a1f5fc0e2201f5570bccaa3deb3"
}
After I change the value of these attributes or add new keys, I run configure recipes on my already existing instances, hoping that it will pick up the new stack settings JSON, but it doesnt work. Am I missing something?
PS. I am new to Opswork , so probably there is a more elegant way to do this.
You may need to run update_cookbooks on already existing instances (or do a deploy). (Stack -> Run Command)
Here's an experiment: do new instances read these values correctly / configure themselves with these values? If so, update_cookbooks should help you out.