AWS CLI Update_Stack can't pass parameter value containing a / - amazon-web-services

I've been banging my head all morning on trying to create a powershell script that will ultimately update an AWS stack. Everything is great right up to the point where I have to pass parameters to the cloudformation template.
One of the parameter values (ParameterKey=ZipFilePath) contains a /. But the script fails complaining that it was expecting a = but found a /. I've tried escaping the slash but then the API complains that it found the backslash instead of an equals. Where am I going wrong?
... <snip creating a zip file> ...
$filename = ("TotalCommApi-" + $DateTime + ".zip")
aws s3 cp $filename ("s3://S3BucketName/TotalCommApi/" + $filename)
aws cloudformation update-stack --stack-name TotalCommApi-Dev --template-url https://s3-region.amazonaws.com/S3bucketName/TotalCommApi/TotalCommApiCFTemplate.json --parameters ParameterKey=S3BucketName,ParameterValue=S3BucketNameValue,UsePreviousValue=false ParameterKey=ZipFilePath,ParameterValue=("TotalCommApi/" + $filename) ,UsePreviousValue=false
cd C:\Projects\TotalCommApi\TotalComm_API
And here is the pertinent section from the CloudFormation Template:
"Description": "An AWS Serverless Application that uses the ASP.NET Core framework running in Amazon Lambda.",
"Parameters": {
"ZipFilePath": {
"Type": "String",
"Description": "Path to the zip file containing the Lambda Functions code to be published."
},
"S3BucketName": {
"Type": "String",
"Description": "Name of the S3 bucket where the ZipFile resides."
}
},
"AWSTemplateFormatVersion": "2010-09-09",
"Outputs": {},
"Conditions": {},
"Resources": {
"ProxyFunction": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": {"Ref": "S3BucketName" },
"S3Key": { "Ref": "ZipFilePath" }
},
And this is the error message generated by PowerShell ISE
[image removed]
Update: I am using Windows 7 which comes with Powershell 2. I updgraded to Powershell 4. Then my script yielded this error:
On recommendation from a consulting firm, I uninstalled the CLI that I installed via msi, then I upgraded Python to 3.6.2 and then re-installed the CLI via pip. I still get the same error. I "echo"d the command to the screen and this is what I see:
upload: .\TotalCommApi-201806110722.zip to s3://S3bucketName/TotalCommApi/TotalCommApi-201806110722.zip
aws
cloudformation
update-stack
--stack-name
TotalCommApi-Dev
--template-url
https://s3-us-west-2.amazonaws.com/s3BucketName/TotalCommApi/TotalCommApiCFTemplate.json
--parameters
ParameterKey=S3BucketName
UsePreviousValue=true
ParameterKey=ZipFilePath
ParameterValue=TotalCommApi/TotalCommApi-201806110722.zip

Sorry for the delay getting back to you on this - the good news is that I might have a hint about what your issue is.
ParameterKey=ZipFilePath,ParameterValue=("TotalCommApi/" + $filename) ,UsePreviousValue=false
I was driving myself mad trying to reproduce this issue. Why? Because I assumed that the space after ("TotalCommApi/" + $filename) was an artifact from copying, not the actual value that you were using. When I added the space in:
aws cloudformation update-stack --stack-name test --template-url https://s3.amazonaws.com/test-bucket-06-09/test.template --parameters ParameterKey=S3BucketName,ParameterValue=$bucketname,UsePreviousValue=false ParameterKey=ZipFilePath,ParameterValue=testfolder/$filename ,UsePreviousValue=false
Error parsing parameter '--parameters': Expected: '=', received: ','
This isn't exactly your error message (, instead of /), but I think it's probably a similar issue in your case - check to make sure the values that are being used in your command don't have extra spaces somewhere.

Related

Running a shell script in CloudFormation cfn-init

I am trying to run a script in the cfn-init command but it keeps timing out.
What am I doing wrong when running the startup-script.sh?
"WebServerInstance" : {
"Type" : "AWS::EC2::Instance",
"DependsOn" : "AttachGateway",
"Metadata" : {
"Comment" : "Install a simple application",
"AWS::CloudFormation::Init" : {
"config" : {
"files": {
"/home/ec2-user/startup_script.sh": {
"content": {
"Fn::Join": [
"",
[
"#!/bin/bash\n",
"aws s3 cp s3://server-assets/startserver.jar . --region=ap-northeast-1\n",
"aws s3 cp s3://server-assets/site-home-sprint2.jar . --region=ap-northeast-1\n",
"java -jar startserver.jar\n",
"java -jar site-home-sprint2.jar --spring.datasource.password=`< password.txt` --spring.datasource.username=`< username.txt` --spring.datasource.url=`<db_url.txt`\n"
]
]
},
"mode": "000755"
}
},
"commands": {
"start_server": {
"command": "./startup_script.sh",
"cwd": "~",
}
}
}
}
},
The file part works fine and it creates the file but it times out at running the command.
What is the correct way of executing a shell script?
You can tail the logs in /var/log/cfn-init.log and detect the issues while running the script.
The commands in Cloudformation Init are ran as sudo user by default. Maybe there can be an issue were your script is residing in /home/ec2-user/ and you are trying to run the script from '~' (i.e. /root).
Please give the absolute path (/home/ec2-user) in cwd. It will solve your concern.
However, the exact issue can be fetched from the logs only.
Usually the init scripts are executed by root unless specified otherwise. Can you try giving the full path while running your startup script. You can give cloudkast a try. It is an online cloudformation template generator. Makes easier creating objects such as aws::cloudformation::init.

How pass json value for admin-update-user-attributes operation via cli in aws?

Consider the example:
aws cognito-idp admin-update-user-attributes --user-pool-id myUserPollId
--username myUser
--user-attributes [{"Name": "custom:roles","Value": "ROLE1,ROLE2"}] --region us-east-1
This gets me error:
Invalid JSON:
[{Name:
You can always try using shorthand syntax:
--user-attributes Name="custom:roles",Value="ROLE1,ROLE2"
If you really want to use the JSON syntax, try this:
--user-attributes '[{"Name" : "custom:roles","Value" : "ROLE1,ROLE2"}]'
Ensure that the user-attributes list is enclosed in single quotes
--user-attributes '[{"Name": "phone_number", "Value": "+123434532"},
{"Name": "name", "Value":"name_your"}]'
In case someone get stuck in the same problem again, below are the tested steps to have user attributes updated via aws cli with json file.
Step 0: Setup AWS CLI in case you haven't already. Mac users can run:
brew install awscli
Step 1: Have a valid json handy with you, saved in a file. Sample json with valid format:
{
"UserAttributes": [{
"Name": "custom:additional-attribute1",
"Value": "Value for additional attribute 1"
},
{
"Name": "custom:additional-attribute2",
"Value": "Value for additional attribute 2"
}
]
}
Step 2: Run the following in your console:
aws cognito-idp admin-update-user-attributes --user-pool-id XX-XXXX-X_XXXXXXXXX --username XXXXX#XXXXX.com --cli-input-json file:///Users/YOUR_PATH_TO_THE_FILE/user-attributes.json
Parameters:
--user-pool-id :: Your user pool ID.
--username :: The user you want to udpate.
--cli-input-json :: This is the command that loads json file and parses it.
That's it. If your json is valid and aws cli authorises, the given user record should be updated instantly.

AWS put-bucket-notification-configuration won't let me use "Filter"

The command I use:
aws s3api put-bucket-notification-configuration --bucket bucket-name --notification-configuration file:///Users/chris/event_config.json
Works fine if I take out the "Filter" key. As soon as I add it in, I get:
Parameter validation failed:
Unknown parameter in NotificationConfiguration.LambdaFunctionConfigurations[0]: "Filter", must be one of: Id, LambdaFunctionArn, Events
Here's my JSON file:
{
"LambdaFunctionConfigurations": [
{
"LambdaFunctionArn": "arn:aws:lambda:us-east-1:000000000:function:name",
"Events": [
"s3:ObjectCreated:*"
],
"Filter": {
"Key": {
"FilterRules": [
{
"Name": "prefix",
"Value": "images/"
}
]
}
}
}
]
}
When I look at the command's docs (http://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-notification-configuration.html), I don't see any mistake. I've tried copy/pasting, carefully looking over, etc... Any help would be greatly appreciated!
You need to be running at least version 1.7.46 of aws-cli, released 2015-08-20.
This release adds Amazon S3 support for event notification filters and fixes some issues.
https://aws.amazon.com/releasenotes/CLI/3585202016507998
The aws-cli utility contains a lot of built-in intelligence and validation logic. New features often require the code in aws-cli to be updated, and Filter on S3 event notifications is a relatively recent feature.
See also: https://aws.amazon.com/blogs/aws/amazon-s3-update-delete-notifications-better-filters-bucket-metrics/

Publish S3 Bucket Notification to SQS

I am trying to set up my S3 to notify my SQS Queue for a "PUT" Object Creation Event.
I am able to achieve this using CLI by:
aws --profile QA s3api put-bucket-notification --bucket <BUCKET_NAME> --notification-configuration '{ "QueueConfiguration": { "Id": "<EVENT ID>", "Event": "s3:ObjectCreated:Put", "Queue": "<QUEUE ARN>" } }'
Also able to do the same using Java:
NotificationConfiguration notificationConfiguration = new QueueConfiguration(queueArn, EnumSet.of(S3Event.ObjectCreatedByPut));
BucketNotificationConfiguration bucketNotificationConfiguration = new BucketNotificationConfiguration("DropShipInboundQueueDelivery", notificationConfiguration);
client.setBucketNotificationConfiguration(bucketName, bucketNotificationConfiguration)
However when I tried to something similar using CloudFormation template, I cannot find any way to trigger a notification to SQS. The only option I see that works and is documented is to trigger notification to SNS.
I have referred the Cloud Formation Documentation:
I looked at the AWS::S3::Bucket docs to look at the outer syntax. I saw NotificationConfiguration which I need to set
However the Notification Configuration can only contain a list of TopicConfigurations with was the old constructor in JDK before QueueConfiguration was supported
I tried doing something like this:
"NotificationConfiguration" :{
"QueueConfiguration": {
"Id": "DropshipInboundEventNotification",
"Event": "s3:ObjectCreated:Put",
"Queue": "arn:aws:sqs:*:*:Dropship-Inbound-qa"
}
},
But this as expected threw an error: "Encountered unsupported property QueueConfiguration" from amazon.
Looked at this API documentation
I would like to know if someone has been able to do this using CloudFormation Templates as thats how I am maintaining all the other AWS resources and do not want to do anything special for this particular feature.
Any help is appreciated.
There is no need "Id" in Cloudformation Template ( You can check from QueueConfiguration Doc ) and your second mistake, that is not "QueueConfiguration", it's "QueueConfigurations". Because of that you get an error that says "Encountered unsupported property QueueConfiguration"
It must be something like that.
"S3Bucket":{
"Type" : "AWS::S3::Bucket",
"Properties" : {
"AccessControl" : String,
"BucketName" : String,
"CorsConfiguration" : CORS Configuration,
"LifecycleConfiguration" : Lifecycle Configuration,
"LoggingConfiguration" : Logging Configuration,
"NotificationConfiguration" :
{ "QueueConfigurations" : [ {
"Event" : "s3:ObjectCreated:Put",
"Queue" : "arn:YOURQUEUEARN"
} ] },
"Tags" : [ Resource Tag, ... ],
"VersioningConfiguration" : Versioning Configuration,
"WebsiteConfiguration" : Website Configuration Type
}
}
While you are reading cloudformation template documents, you must be careful about "Required:" sections. If it is not required, you don't need to fill it, just remove that line from your template if you don't use it( Like S3 Tags ).
Other Docs about it:
S3BucketDocs
NotificationConfigurationDocs

Map 'ec2-register snapshot' syntax onto 'register-image AMI 'syntax using awscli

What is the correct syntax for mapping a snapshot onto an AMI using awscli?
More explicitly, how do I map the old syntax
'ec2-register -s snap-9abc1234 --kernel 99abcdef' onto the new syntax
'aws ec2 --register-image' ?
It's the following:
aws ec2 register-image --kernel-id <your-kernel> --root-device-name /dev/sda1 --block-device-mappings [list in JSON shown below]
[
{
"VirtualName": "string",
"DeviceName": "string",
"Ebs": {
"SnapshotId": "string",
"VolumeSize": integer,
"DeleteOnTermination": true|false,
"VolumeType": "standard"|"io1",
"Iops": integer
},
"NoDevice": "string"
}
...
]
You can run aws ec2 register-image help for help on the command.
Make sure you are using the awscli python package on not the aws package as that one is different (not the official one)
Here's a link to the github repo:
https://github.com/aws/aws-cli