I am trying to delete all resources in my aws account, but the directions for aws-nuke says I need to create a config file:
At first you need to create a configfile for aws-nuke. This is a minimal one:
regions:
- eu-west-1
- global
account-blacklist:
- "999999999999" # production
accounts:
"000000000000": {} # aws-nuke-example
With this config we can run aws-nuke:
My question is, how do I create this config file that deletes everything associated with an account and leaves me with a blank account? Thanks!
If you want to completely nuke everything associated with an account you just have to replace the zeros for the account number you want to erase like in your example. The {} means all resources types. Save the file as.YAML format and next issue the command like this:
aws-nuke -c config/example.yaml --profile demo
Check my example config/example.yaml file here:
regions:
#Regions where the resources are
- "global"
- "eu-central-1"
- "eu-west-1"
- "eu-west-2"
- "eu-east-1"
- "eu-east-2"
- "us-east-1"
- "us-east-2"
- "us-west-1"
- "us-west-2"
account-blocklist:
#Accounts you dont want to change
- 123456789101 # e.g production account
resource-types: #not mandatory
targets:
# Specific resources you want to remove
- S3Object
- S3Bucket
- EC2Volume
excludes: #not mandatory
# Specific resources you don't want to remove
- IAMUser
accounts:
943725333913: {}
# the {} means all resources associated with this account
# instead you can use filters like this:
943725333913:
filters:
S3Bucket:
- "s3://my-bucket"
S3Object:
- type: "glob"
value: "s3://my-bucket/*"
Related
I have stored a key in the Secret manager of GCP and I'm trying to use that secret in the cloudbuild.yaml but every time I have this error:
ERROR: (gcloud.functions.deploy) argument --set-secrets: Secrets value configuration must match the pattern 'SECRET:VERSION' or 'projects/{PROJECT}/secrets/{SECRET}:{VERSION}' or 'projects/{PROJECT}/secrets/{SECRET}/versions/{VERSION}' where VERSION is a number or the label 'latest' [ 'projects/gcp-project/secrets/SECRETKEY/versions/latest' ]]
My cloud build file looks like this:
steps:
- id: installing-dependencies
name: 'python'
entrypoint: pip
args: ["install", "-r", "src/requirements.txt", "--user"]
- id: deploy-function
name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
args:
- gcloud
- functions
- deploy
- name_of_my_function
- --region=us-central1
- --source=./src
- --trigger-topic=name_of_my_topic
- --runtime=python37
- --set-secrets=[ SECRETKEY = 'projects/gcp-project/secrets/SECRETKEY/versions/latest' ]
waitFor: [ "installing-dependencies" ]
I was reading the documentation, but I don't have any other clue that could help me.
As mentioned by al-dann, there should not be any space in set-secret line as you can see the documentation
Final correction in code :
--set-secrets=[SECRETKEY='projects/gcp-project/secrets/SECRETKEY/versions/latest']
For more information, you can refer to the stackoverflow thread and blog where brief information about secret manager has been well explained.
I am using the official helm chart of Jenkins.
I have enabled backup and also provided backup credentials
Here is the relevant config in values.yaml
## Backup cronjob configuration
## Ref: https://github.com/maorfr/kube-tasks
backup:
# Backup must use RBAC
# So by enabling backup you are enabling RBAC specific for backup
enabled: true
# Used for label app.kubernetes.io/component
componentName: "jenkins-backup"
# Schedule to run jobs. Must be in cron time format
# Ref: https://crontab.guru/
schedule: "0 2 * * *"
labels: {}
annotations: {}
# Example for authorization to AWS S3 using kube2iam
# Can also be done using environment variables
# iam.amazonaws.com/role: "jenkins"
image:
repository: "maorfr/kube-tasks"
tag: "0.2.0"
# Additional arguments for kube-tasks
# Ref: https://github.com/maorfr/kube-tasks#simple-backup
extraArgs: []
# Add existingSecret for AWS credentials
existingSecret: {}
# gcpcredentials: "credentials.json"
## Example for using an existing secret
# jenkinsaws:
## Use this key for AWS access key ID
awsaccesskey: "AAAAJJJJDDDDDDJJJJJ"
## Use this key for AWS secret access key
awssecretkey: "frkmfrkmrlkmfrkmflkmlm"
# Add additional environment variables
# jenkinsgcp:
## Use this key for GCP credentials
env: []
# Example environment variable required for AWS credentials chain
# - name: "AWS_REGION"
# value: "us-east-1"
resources:
requests:
memory: 1Gi
cpu: 1
limits:
memory: 1Gi
cpu: 1
# Destination to store the backup artifacts
# Supported cloud storage services: AWS S3, Minio S3, Azure Blob Storage, Google Cloud Storage
# Additional support can added. Visit this repository for details
# Ref: https://github.com/maorfr/skbn
destination: "s3://jenkins-data/backup"
However the backup job fails as follows:
2020/01/22 20:19:23 Backup started!
2020/01/22 20:19:23 Getting clients
2020/01/22 20:19:26 NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
What is missing?
you must create secret which looks like this:
kubectl create secret generic jenkinsaws --from-literal=jenkins_aws_access_key=ACCESS_KEY --from-literal=jenkins_aws_secret_key=SECRET_KEY
then consume it like this:
existingSecret:
jenkinsaws:
awsaccesskey: jenkins_aws_access_key
awssecretkey: jenkins_aws_secret_key
where jenkins_aws_access_key/jenkins_aws_secret_key it's key of the secret
backup:
enabled: true
destination: "s3://jenkins-pumbala/backup"
schedule: "15 1 * * *"
env:
- name: "AWS_ACCESS_KEY_ID"
value: "AKIDFFERWT***D36G"
- name: "AWS_SECRET_ACCESS_KEY"
value: "5zGdfgdfgdf***************Isi"
I'm using deployment manager to set the IAM policy of an existing pub/sub topic- I don't want to acquire it and I cannot create it with deployment manager (because it exists). So I want to set a policy on an existing resource
I can do this with buckets but the docs are confusing and I can't find the right methods for buckets
I want to do this (resource level bindings) for a topic instead of bucket:
resources:
- name: mybucket
action: gcp-types/storage-v1:storage.buckets.setIamPolicy
properties:
bucket: mybucket
bindings:
- role: roles/storage.admin
members:
- "serviceAccount:sdfsfds#sdfsdf.com"
I can only find gcp-types/pubsub-v1:projects.topics.setIamPolicy which seems like its at the project level? What is the right api for setting an IAM policy on a specific topic?
The google APIs seem inconsistent here- are these too methods equivalent? Docs are confusing:
https://cloud.google.com/storage/docs/json_api/v1/buckets/setIamPolicy
https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.topics/setIamPolicy
I attempted this but getting an error:
- name: mytopic
action: gcp-types/pubsub-v1:pubsub.projects.topics.setIamPolicy
properties:
resource: mytopic
bindings:
- role: roles/pubsub.admin
members:
- "serviceAccount:ssdfsdf#sdfsdf.com"
Getting error:
message: '{"ResourceType":"gcp-types/pubsub-v1:pubsub.projects.topics.setIamPolicy","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Invalid
JSON payload received. Unknown name \"bindings\": Cannot find field.","status":"INVALID_ARGUMENT","details":[{"#type":"type.googleapis.com/google.rpc.BadRequest","fieldViolations":[{"description":"Invalid
JSON payload received. Unknown name \"bindings\": Cannot find field."}]}],"statusMessage":"Bad
Request","requestPath":"https://pubsub.googleapis.com/v1/projects/myproject/topics/mytopic:setIamPolicy","httpMethod":"POST"}}
When I tried projects.topics.setIamPolicy I got:
- code: COLLECTION_NOT_FOUND
message: Collection 'projects.topics.setIamPolicy' not found in discovery doc 'https://pubsub.googleapis.com/$discovery/rest?version=v1'
The pubsub-v1:projects.topics.setIamPolicy is at the topic level and the https://iam.googleapis.com/v1/{resource=projects/*/serviceAccounts/*}:setIamPolicy is to set the a Pub/Sub or other resources at the project level.
You get those error because you are giving Pub/Sub admin and this is a role at the project level. The example roles you can provide are:
roles/viewer
roles/editor
roles/owner
I understand that you are trying to to deploy a topic having a IAM policy that allows only one service account to a topic. You have to use a yaml file and a python file if that is the environment you are using.
In the python file you will set the IAM for the topic with the method "set_iam_policy" which takes 2 arguments, the policy and the topic path:
client = pubsub_v1.PublisherClient()
topic_path = client.topic_path(project, topic_name)
policy = client.get_iam_policy(topic_path)
# Add all users as viewers.
policy.bindings.add(
role='roles/pubsub.viewer',
members=['allUsers'])
# Add a group as a publisher.
policy.bindings.add(
role='roles/pubsub.publisher',
members=['group:cloud-logs#google.com'])
# Set the policy
policy = client.set_iam_policy(topic_path, policy)
print('IAM policy for topic {} set: {}'.format(
topic_name, policy))
For deployment manager:
imports:
- path: templates/pubsub/pubsub.py
name: pubsub.py
resources:
- name: test-pubsub
type: pubsub.py
properties:
topic: test-topic
accessControl:
- role: roles/pubsub.subscriber
members:
- user:demo#user.com
subscriptions:
- name: first-subscription
accessControl:
- role: roles/pubsub.subscriber
members:
- user:demo#user.com
- name: second-subscription
ackDeadlineSeconds: 15
I want to create a SecretsManager secret with two values both generated on the fly. Is that possible and how do I do it?
For example, if I wanted the final secret to look like this in JSON, what would the CloudFormation yaml look like?
{
"password": "<Generated Value>",
"serviceId": "fd07f2ab-96bd-4c5c-a4a9-9b8c43b666d7",
"login": "<Different Generated Value>"
}
If this is part of a CF template that generates the password part of the secret string, how would I generate a dynamic 'login' field as well?
#This is a Secret resource with a randomly generated password in its SecretString JSON.
MySecretA:
Type: 'AWS::SecretsManager::Secret'
Properties:
Name: MySecretForAppA
Description: "This secret has a dynamically generated secret password and login."
GenerateSecretString:
SecretStringTemplate: '{"service-id": "<some guid>"}'
GenerateStringKey: "password"
PasswordLength: 30
ExcludeCharacters: '"#/\'
It appears that the AWS::SecretsManager::Secret resource only generates a single secret.
I could not find a way to refer to the actual contents of the secret via !Ref, which would have allowed two secrets to be generated, then referenced and combined in another resource. The only !Ref value available is the ARN of the secret.
Thus, I couldn't see a way to create two secrets in the one resource.
The AWS::SecretsManager::Secret resource does not support generating multiple random keys in a single secret. One way to accomplish this today is to leverage a custom CloudFormation resource (see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html). You can write a custom resource lambda that generates two random strings and does a PutSecretValue on the secret with the new values.
Another possible option is to have CloudFormation generate two secrets (one for username and one for password). I would not recommend this approach since every time you need the secret you will need to do two retrieves and it doubles your AWS Secrets Manager costs.
You could create two secrets, and only use one.
For example, you could create one for the password:
MySecretPassword:
Type: 'AWS::SecretsManager::Secret'
Properties:
Name: MySecretPasswordForAppA
Description: "This secret has a dynamically generated secret password."
GenerateSecretString:
SecretStringTemplate: '{"serviceId": "<some guid>"}'
GenerateStringKey: "password"
PasswordLength: 30
ExcludeCharacters: '"#/\'
And then another one that creates the login value, and uses the generated value guarded by MySecretPassword:
MySecret:
Type: 'AWS::SecretsManager::Secret'
Properties:
Name: MySecretForAppA
Description: "This secret has a dynamically generated secret password and login."
GenerateSecretString:
SecretStringTemplate:
!Join:
- ''
- '{ "password": "'
- !Join:
- ':'
- '{{resolve'
- 'secretsmanager'
- !Ref: MySecretPassword
- 'SecretString'
- 'password}}'
- '", "service-id": "'
- !Join:
- ':'
- '{{resolve'
- 'secretsmanager'
- !Ref: MySecretPassword
- 'SecretString'
- 'serviceId}}'
- '" }'
GenerateStringKey: "login"
PasswordLength: 30
ExcludeCharacters: '"#/\'
I am trying to run aws-nuke to delete all the resources.
I am trying to run command
aws-nuke -c config/example.yaml --profile demo
config/example.yaml
---
regions:
- "global" # This is for all global resource types e.g. IAM
- "eu-west-1"
account-blacklist:
- "999999999999" # production
# optional: restrict nuking to these resources
resource-types:
targets:
- IAMUser
- IAMUserPolicyAttachment
- IAMUserAccessKey
- S3Bucket
- S3Object
- Route53HostedZone
- EC2Instance
- CloudFormationStack
accounts:
555133742123#demo:
filters:
IAMUser:
- "admin"
IAMUserPolicyAttachment:
- property: RoleName
value: "admin"
IAMUserAccessKey:
- property: UserName
value: "admin"
S3Bucket:
- "s3://my-bucket"
S3Object:
- type: "glob"
value: "s3://my-bucket/*"
Route53HostedZone:
- property: Name
type: "glob"
value: "*.zone.loc."
CloudFormationStack:
- property: "tag:team"
value: "myTeam"
Errors screenshot below.What is this missing
Disclaimer: I am an author of aws-nuke.
This is not an configuration problem of your YAML file, but a missing setting in your AWS account.
The IAM Alias is a globally unique name for your AWS Account. aws-nuke requires this as a safety guard, so you do not accidentally destroy your production accounts. The idea is that every production account contains at least the substring prod.
This might sound a bit unnecessary to demand this account, but we are very passionate to not nuke any production account.
You can follow the docs to specify the Alias via the web console, or you use the CLI:
aws iam create-account-alias --profile demo --account-alias my-test-account-8gmst3`
I guess we need to improve the error message.