Since PR: https://github.com/aws/aws-cdk/pull/19454 and release v2.18.0 CDK is supposed to support SSO credentials via the AWS CLI v2 profiles.
However no matter what I do I simply cannot get this to work.
I have created a request for updated documentation in the AWS CDK Issues section, since no official documentation explains how this is supposed to function in practice, and the official documentation still saying it is not supported and to use yawsso: https://github.com/aws/aws-cdk/issues/21314
From going through 4 years of old threads to now I have attempted the following settings with 0 success.
My .aws/config file (sensitive values redacted):
[profile DEV-NN-HSMX]
sso_start_url = https://my-company-url.awsapps.com/start#/
sso_region = eu-central-1
sso_account_name = MY-ACCOUNT
sso_account_id = MY-ACCOUNT-ID
sso_role_name = AdministratorAccess
region = eu-central-1
Running aws sso login --profile "DEV-NN-HSMX" redirects me as expected and I can authenticate with my SSO provider.
Running aws sts get-caller-identity --profile "DEV-NN-HSMX" works as expected and confirms my SSO identity.
Running aws s3 ls --profile "DEV-NN-HSMX" works as expected and shows that the credentials have access.
When attempting to run any CDK commands however. I simply cannot make it work.
AWS CLI version: 2.7.16
AWS CDK version: 2.33.0
I have attempted a combination of all the following, either separately, mixed in all combinations and all at once.
cdk deploy --profile "DEV-NN-HSMX"
Exporting both the $AWS_PROFILE and/or the $CDK_DEFAULT_PROFILE environment variables:
cdk doctor
ℹ️ CDK Version: 2.33.0 (build 859272d)
ℹ️ AWS environment variables:
- AWS_CA_BUNDLE = /home/vscode/certs/cacert.pem
- AWS_PROFILE = DEV-NN-HSMX
- AWS_REGION = eu-central-1
- AWS_STS_REGIONAL_ENDPOINTS = regional
- AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
- AWS_SDK_LOAD_CONFIG = 1
ℹ️ CDK environment variables:
- CDK_DEFAULT_PROFILE = DEV-NN-HSMX
- CDK_DEFAULT_REGION = eu-central-1
I have tried with a deleted .aws/credentials file as well as one that is just empty.
I have deleted everything in aws\sso\cache and in .aws\cli\cache to make sure no expired credential information remained and then re-authenticated with aws sso login --profile "DEV-NN-HSMX".
If I use yawsso --profiles DEV-NN-HSMX and get temporary credentials into .aws/credentials for my profile, it works fine.
I have been able to bootstrap and deploy without issues using the credential conversion. Proving that from a connection, access rights and bootstrap standpoint everything works as expected.
When using any of the SSO methods as explained above without exporting credentials, I always get the following error message.
cdk deploy --profile "DEV-NN-HSMX"
✨ Synthesis time: 4.18s
Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
Running the command with full verbosity gives this output:
cdk deploy --trace --verbose --profile "DEV-NN-HSMX"
CDK toolkit version: 2.33.0 (build 859272d)
Command line arguments: {
_: [ 'deploy' ],
trace: true,
verbose: 1,
v: 1,
profile: 'DEV-NN-HSMX',
defaultProfile: 'DEV-NN-HSMX',
defaultRegion: 'eu-central-1',
lookups: true,
'ignore-errors': false,
ignoreErrors: false,
json: false,
j: false,
debug: false,
ec2creds: undefined,
i: undefined,
'version-reporting': undefined,
versionReporting: undefined,
'path-metadata': true,
pathMetadata: true,
'asset-metadata': true,
assetMetadata: true,
'role-arn': undefined,
r: undefined,
roleArn: undefined,
staging: true,
'no-color': false,
noColor: false,
ci: false,
all: false,
'build-exclude': [],
E: [],
buildExclude: [],
execute: true,
force: false,
f: false,
parameters: [ {} ],
'previous-parameters': true,
previousParameters: true,
logs: true,
'$0': '/home/vscode/.local/state/fnm_multishells/216_1658735050827/bin/cdk'
}
cdk.json: {
"app": "npx ts-node --prefer-ts-exts bin/cdk-demo.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"#aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"#aws-cdk/core:stackRelativeExports": true,
"#aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"#aws-cdk/aws-lambda:recognizeVersionProps": true,
"#aws-cdk/aws-lambda:recognizeLayerVersion": true,
"#aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true,
"#aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
"#aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
"#aws-cdk/core:checkSecretUsage": true,
"#aws-cdk/aws-iam:minimizePolicies": true,
"#aws-cdk/core:validateSnapshotRemovalPolicy": true,
"#aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
"#aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
"#aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
"#aws-cdk/core:target-partitions": [
"aws",
"aws-cn"
]
}
}
merged settings: {
versionReporting: true,
pathMetadata: true,
output: 'cdk.out',
app: 'npx ts-node --prefer-ts-exts bin/cdk-demo.ts',
watch: {
include: [ '**' ],
exclude: [
'README.md',
'cdk*.json',
'**/*.d.ts',
'**/*.js',
'tsconfig.json',
'package*.json',
'yarn.lock',
'node_modules',
'test'
]
},
context: {
'#aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
'#aws-cdk/core:stackRelativeExports': true,
'#aws-cdk/aws-rds:lowercaseDbIdentifier': true,
'#aws-cdk/aws-lambda:recognizeVersionProps': true,
'#aws-cdk/aws-lambda:recognizeLayerVersion': true,
'#aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
'#aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
'#aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
'#aws-cdk/core:checkSecretUsage': true,
'#aws-cdk/aws-iam:minimizePolicies': true,
'#aws-cdk/core:validateSnapshotRemovalPolicy': true,
'#aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName': true,
'#aws-cdk/aws-s3:createDefaultLoggingPolicy': true,
'#aws-cdk/aws-sns-subscriptions:restrictSqsDescryption': true,
'#aws-cdk/core:target-partitions': [ 'aws', 'aws-cn' ]
},
debug: false,
assetMetadata: true,
profile: 'DEV-NN-HSMX',
toolkitBucket: {},
staging: true,
bundlingStacks: [ '*' ],
lookups: true
}
Using CA bundle path: /home/vscode/certs/cacert.pem
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to eu-central-1
Resolving default credentials
Could not refresh notices: Error: unable to get local issuer certificate
Unable to determine the default AWS account: ProcessCredentialsProviderFailure: Profile DEV-NN-HSMX did not include credential process
at ProcessCredentials2.load (/home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials/process_credentials.js:102:11)
at ProcessCredentials2.coalesceRefresh (/home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials.js:205:12)
at ProcessCredentials2.refresh (/home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials/process_credentials.js:163:10)
at ProcessCredentials2.get2 [as get] (/home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials.js:122:12)
at resolveNext2 (/home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials/credential_provider_chain.js:125:17)
at /home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials/credential_provider_chain.js:126:13
at /home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials.js:124:23
at /home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-sdk/lib/credentials.js:212:15
at processTicksAndRejections (node:internal/process/task_queues:78:11) {
code: 'ProcessCredentialsProviderFailure',
time: 2022-07-25T15:01:41.645Z
}
context: {
'#aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
'#aws-cdk/core:stackRelativeExports': true,
'#aws-cdk/aws-rds:lowercaseDbIdentifier': true,
'#aws-cdk/aws-lambda:recognizeVersionProps': true,
'#aws-cdk/aws-lambda:recognizeLayerVersion': true,
'#aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
'#aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
'#aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
'#aws-cdk/core:checkSecretUsage': true,
'#aws-cdk/aws-iam:minimizePolicies': true,
'#aws-cdk/core:validateSnapshotRemovalPolicy': true,
'#aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName': true,
'#aws-cdk/aws-s3:createDefaultLoggingPolicy': true,
'#aws-cdk/aws-sns-subscriptions:restrictSqsDescryption': true,
'#aws-cdk/core:target-partitions': [ 'aws', 'aws-cn' ],
'aws:cdk:enable-path-metadata': true,
'aws:cdk:enable-asset-metadata': true,
'aws:cdk:version-reporting': true,
'aws:cdk:bundling-stacks': [ '*' ]
}
outdir: cdk.out
env: {
CDK_DEFAULT_REGION: 'eu-central-1',
CDK_CONTEXT_JSON: '{"#aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId":true,"#aws-cdk/core:stackRelativeExports":true,"#aws-cdk/aws-rds:lowercaseDbIdentifier":true,"#aws-cdk/aws-lambda:recognizeVersionProps":true,"#aws-cdk/aws-lambda:recognizeLayerVersion":true,"#aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021":true,"#aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver":true,"#aws-cdk/aws-ec2:uniqueImdsv2TemplateName":true,"#aws-cdk/core:checkSecretUsage":true,"#aws-cdk/aws-iam:minimizePolicies":true,"#aws-cdk/core:validateSnapshotRemovalPolicy":true,"#aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName":true,"#aws-cdk/aws-s3:createDefaultLoggingPolicy":true,"#aws-cdk/aws-sns-subscriptions:restrictSqsDescryption":true,"#aws-cdk/core:target-partitions":["aws","aws-cn"],"aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true,"aws:cdk:version-reporting":true,"aws:cdk:bundling-stacks":["*"]}',
CDK_OUTDIR: 'cdk.out',
CDK_CLI_ASM_VERSION: '20.0.0',
CDK_CLI_VERSION: '2.33.0'
}
✨ Synthesis time: 4.54s
Reading existing template for stack CdkDemoStack.
Reading cached notices from /home/vscode/.cdk/cache/notices.json
Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
Error: Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
at SdkProvider.resolveEnvironment (/home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-cdk/lib/api/aws-auth/sdk-provider.ts:238:13)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at CloudFormationDeployments.prepareSdkFor (/home/vscode/.local/share/fnm/node-versions/v16.16.0/installation/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:432:33)
I do notice the ProcessCredentialsProviderFailure in the output, but this is not very informative on how to solve it.
Anyone have any ideas or input?
It seems like agnostic stacks where you do not put the environment information directly into the stack code does not work with the new SSO integration.
Adding the environment information into the stack code makes it work:
const app = new cdk.App();
new CdkDemoStack(app, 'CdkDemoStack', {
env: { account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION
},
});
So I have a nodejs webservice which I push into Cloud Foundry (PCF), then I am storing some credentials in Vault so when a user hits my web service endpoint with some credentials I extract the credentials from the Vault, compare them against the credentials from the request and if the match I allow the request to be processed else I reject the request.
So to install Vault in PCF I use the next command:
cf create-service hashicorp-vault shared foo-vault
Then I create a key using this command:
create-service-key foo-vault foo-vault-key
Then I bind the service to the app like this:
cf bind-service foo-ws foo-vault
I restage the web service and when I print the environmental variables using this command:
cf restage foo-ws
I get this values:
{
"hashicorp-vault": [{
"credentials": {
"address": "http://somehost:433/",
"auth": {
"accessor": "kMr3iCSlekSN2d1vpPjbjzUk",
"token": "some token"
},
"backends": {
"generic": [
"cf/7f1a12a9-4a52-4151-bc96-874380d30182/secret",
"cf/c4073566-baee-48ae-88e9-7c7c7e0118eb/secret"
],
"transit": [
"cf/7f1a12a9-4a52-4151-bc96-874380d30182/transit",
"cf/c4073566-baee-48ae-88e9-7c7c7e0118eb/transit"
]
},
"backends_shared": {
"organization": "cf/8d4b992f-cca3-4876-94e0-e49170eafb67/secret",
"space": "cf/bdace353-e813-4efb-8122-58b9bd98e3ab/secret"
}
},
"label": "hashicorp-vault",
"name": "my-vault",
"plan": "shared",
"provider": null,
"syslog_drain_url": null,
"tags": [],
"volume_mounts": []
}]
}
So my question is if there is a way to define the backends, token and address?
Thanks in advance for your help.
Greetings
Similar to this question How to get Task ID from within ECS container? but I want to get the TaskId for my Fargate task. How can you do this? Like others I want this for logging information.
I'm running a Spring App with ELK stack for logging and would like if possible to include the TaskId in the logs if possible.
Edit
I actually never got this to work by the way, here is my code:
private String getTaskIdInternal() {
String url = System.getenv("ECS_CONTAINER_METADATA_URI_V4") + "/task";
logger.info("Getting ecsMetaDataURL={}", url);
if (url == null) {
throw new RuntimeException("ECS_CONTAINER_METADATA_URI_V4 env variable not defined");
}
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<JsonNode> response = restTemplate.getForEntity(url, JsonNode.class);
logger.info("ecsMetaData={}", response);
JsonNode map = response.getBody();
String taskArn = map.get("TaskARN").asText();
String[] splitTaskArn = taskArn.split("/");
String taskId = splitTaskArn[splitTaskArn.length - 1];
logger.info("ecsTaskId={}", taskId);
return taskId;
}
But I always get this stack trace:
Could not get the taskId from ECS. exception=org.springframework.web.client.HttpClientErrorException: 403 Forbidden
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:118)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:103)
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:732)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:690)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:646)
at org.springframework.web.client.RestTemplate.getForEntity(RestTemplate.java:325)
If you're trying to get the task id in Fargate for ECS you make use of metadata endpoints.
Assuming you're using version 1.4.0 of Fargate you can get this via a http request to ${ECS_CONTAINER_METADATA_URI_V4}/task.
An example response from this endpoint is below
{
"Cluster": "arn:aws:ecs:us-west-2:&ExampleAWSAccountNo1;:cluster/default",
"TaskARN": "arn:aws:ecs:us-west-2:&ExampleAWSAccountNo1;:task/default/febee046097849aba589d4435207c04a",
"Family": "query-metadata",
"Revision": "7",
"DesiredStatus": "RUNNING",
"KnownStatus": "RUNNING",
"Limits": {
"CPU": 0.25,
"Memory": 512
},
"PullStartedAt": "2020-03-26T22:25:40.420726088Z",
"PullStoppedAt": "2020-03-26T22:26:22.235177616Z",
"AvailabilityZone": "us-west-2c",
"Containers": [
{
"DockerId": "febee046097849aba589d4435207c04aquery-metadata",
"Name": "query-metadata",
"DockerName": "query-metadata",
"Image": "mreferre/eksutils",
"ImageID": "sha256:1b146e73f801617610dcb00441c6423e7c85a7583dd4a65ed1be03cb0e123311",
"Labels": {
"com.amazonaws.ecs.cluster": "arn:aws:ecs:us-west-2:&ExampleAWSAccountNo1;:cluster/default",
"com.amazonaws.ecs.container-name": "query-metadata",
"com.amazonaws.ecs.task-arn": "arn:aws:ecs:us-west-2:&ExampleAWSAccountNo1;:task/default/febee046097849aba589d4435207c04a",
"com.amazonaws.ecs.task-definition-family": "query-metadata",
"com.amazonaws.ecs.task-definition-version": "7"
},
"DesiredStatus": "RUNNING",
"KnownStatus": "RUNNING",
"Limits": {
"CPU": 2
},
"CreatedAt": "2020-03-26T22:26:24.534553758Z",
"StartedAt": "2020-03-26T22:26:24.534553758Z",
"Type": "NORMAL",
"Networks": [
{
"NetworkMode": "awsvpc",
"IPv4Addresses": [
"10.0.0.108"
],
"AttachmentIndex": 0,
"IPv4SubnetCIDRBlock": "10.0.0.0/24",
"MACAddress": "0a:62:17:7a:36:68",
"DomainNameServers": [
"10.0.0.2"
],
"DomainNameSearchList": [
"us-west-2.compute.internal"
],
"PrivateDNSName": "ip-10-0-0-108.us-west-2.compute.internal",
"SubnetGatewayIpv4Address": ""
}
]
}
]
}
As you can see you would need to parse the TaskARN to get the TaskID (it is the last part of the ARN if you split by "/".
Amazon do specify the following in the documentation that should be noted.
For tasks using the Fargate launch type and platform versions prior to 1.4.0, the task metadata version 3 and 2 endpoint are supported. For more information, see Task Metadata Endpoint version 3 or Task Metadata Endpoint version 2.
The link in the accepted answer is for EC2 launch type. The direct doc link for Fargate is: https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-metadata-endpoint-v4-fargate.html. The json content seems to be pretty much the same though.
I am trying to deploy a django application through zappa. When I deploy the application everything works correctly and I also get the message "Your updated Zappa deployment is live". But I can't seem to find the url to access the live application.
You can access the url by typing zappa status <stage>, where <stage> is probably something like dev. See also https://github.com/Miserlou/Zappa#status for details.
The printout will have your url as well as other status details of your lambda function.
This works provided you give full permission access to the IAM user (for testing purpose only) though.
I had to configure my zappa_settigs.json by adding these lines:
"apigateway_enabled": true,
"manage_roles": true,
"cors": true,
**so the final zappa settings:**
{
"dev": {
"django_settings": "zappatest.settings",
"apigateway_enabled": true,
"manage_roles": true,
"role_arn": "Role_name",
"role_arn": "arn_name",
"profile_name": "default",
"project_name": "project_name",
"runtime": "python3.8",
"s3_bucket": "bucket_name",
"aws_region": "region_name",
"cors": true,
}
}
I am trying to setup an AWS AMI vagrant provision: http://www.packer.io/docs/builders/amazon-ebs.html
I am using the standard .json config:
{
"type": "amazon-instance",
"access_key": "YOUR KEY HERE",
"secret_key": "YOUR SECRET KEY HERE",
"region": "us-east-1",
"source_ami": "ami-d9d6a6b0",
"instance_type": "m1.small",
"ssh_username": "ubuntu",
"account_id": "0123-4567-0890",
"s3_bucket": "packer-images",
"x509_cert_path": "x509.cert",
"x509_key_path": "x509.key",
"x509_upload_path": "/tmp",
"ami_name": "packer-quick-start {{timestamp}}"
}
It connects fine, and I see it create the instance in my AWS account. However, I keep getting Timeout waiting for SSH as an error. What could be causing this problem and how can I resolve it?
As I mentioned in my comment above this is just because sometimes it takes more than a minute for an instance to launch and be SSH ready.
If you want you could set the timeout to be longer - the default timeout with packer is 1 minute.
So you could set it to 5 minutes by adding the following to your json config:
"ssh_timeout": "5m"