I'm trying to issue an aws s3api put-object command with all arguments specified via the --cli-input-json document, which in this case looks like so:
{
"Body": "this is the part giving me trouble",
"Bucket": "my-bucket",
"Key": "my-key"
}
For the Body property, I can't figure out how to specify a file (on the local system) to put to S3. I've tried both:
"Body": "the_filepath"
"Body": "file://the_filepath"
... but neither work (both result in an Invalid base64 error).
I know I can add the file to the command line call via --body file://the_filepath, but I'm trying to put all command args into the JSON document. I'm also trying to avoid reading in the contents of the object by the controlling script.
I'm stumped and I can't seem to find AWS CLI documentation on this use case.
I am running the below command and it is reporting an error like Invalid JSON:
AWS SSM Command:
aws ssm register-task-with-maintenance-window --window-id mw-06344a0189162e0b3 --targets Key=WindowTargetIds,Values=ae5c621a-17d8-454f-977f-46298f1e6eb8 --task-arn AWS-RunPatchBaseline --service-role-arn arn:aws:iam::xxxxxxxxxxxxx:role/AmazonSSMRoleForInstancesQuickSetup --task-type RUN_COMMAND --max-concurrency 2 --max-errors 1 --priority 1 --task-invocation-parameters '{\"Operation\":{\"Values\":[\"Install\"]}}'
Error:
Error parsing parameter '--task-invocation-parameters': Invalid JSON: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
JSON received: {\"Operation\":{\"Values\":[\"Install\"]}}
I'm using the AWS Documentation.
I used double quotes instead of the single quote but still no charm.
Finally, created an issue on GitHub for AWS Docs.
Kindly assist if you have come across an issue like this before.
The error is saying it wants "double quotes" but your example has single quotes. Maybe the docs have been updated since your posting, but it currently recommends the following format.
Linux/Windows:
--task-invocation-parameters "RunCommand={Parameters={Operation=Install}}"
I'm trying to set up a Custom Domain Name in AWS API Gateway where callers have to specify explicitly the stage name after any base path name. It is something I did in the past but now it seems that, since AWS updated the console interface, it is no more possible.
The final url should be like:
https://example.com/{basePath}/{stage}/function
I tried using the console, but stage is now a mandatory field (chose from a drop-down).
I tried using AWS CLI, but stage is again a mandatory field
aws: error: the following arguments are required: --stage
I tried using Boto3, following the documentation (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/apigateway.html#APIGateway.Client.create_base_path_mapping) but, even if stage can be specified as 'none' (The name of the API's stage that you want to use for this mapping. Specify '(none)' if you want callers to explicitly specify the stage name after any base path name.), doing this returns an error:
botocore.errorfactory.BadRequestException: An error occurred (BadRequestException) when calling the CreateBasePathMapping operation: Invalid stage identifier specified
What is funny (or frustrating) is that I have some custom domain names created with the old console and that are perfectly working, without any stage defined.
It is still possible to specify only the "API ID" and "Path" and leave out the "stage" parameter. I have tried this both from the console and the CLI:
From console: The "Stage" setting is a drop-down as you mentioned, but can be left blank (don't select anything). If you did select a stage, delete the API mapping and add it again
From CLI: Just tried this as well and works fine for me on CLI version aws-cli/1.18.69 Python/3.7.7 Darwin/18.7.0 botocore/1.16.19
$ aws apigateway create-base-path-mapping --domain-name **** --rest-api-id *** --base-path test
{
"basePath": "test",
"restApiId": "***"
}
As there is no create-tag for s3, only put-bucket-tagging can be used, which requires that you include all tags on the resource, not just the new one. Thus there is no way to add a new tag to a bucket that already has tags unless you include all existing tags PLUS your new tag. This makes it way more difficult to use for bulk operations, as you need to get all the tags first, extrapolate it into json, edit the json to add your new tag to every bucket, and then feed that to put-bucket-tagging.
Does anyone have a better way to do this or have a script that does this?
Command I'm trying:
aws s3api put-bucket-tagging --bucket cbe-res034-scratch-29 --tagging "TagSet=[{Key=Environment,Value=Research}]"
Error I get:
An error occurred (InvalidTag) when calling the PutBucketTagging operation: System tags cannot be removed by requester
I get the 'cannot be removed' error because put-bucket-tagging is trying to delete the other 10 tags on this bucket (because I didn't include them in the TagSet) and I don't have access to do so.
You can use resourcegroupstaggingapi to accomplish the result you expect, see it below.
aws resourcegroupstaggingapi tag-resources --resource-arn-list arn:aws:s3:::cbe-res034-scratch-29 --tags Environment=Research
To handle spaces on tag name or value, use it as json.
aws resourcegroupstaggingapi tag-resources --resource-arn-list arn:aws:s3:::cbe-res034-scratch-29 --tags '{"Environment Name":"Research Area"}'
I would strongly recommend using json file instead of command line flags. I have spent few hours yesterday without any success making key and value with white spaces work. This is in the context of Jenkins pipline in groovy calling bash shell script block.
Here is the syntax for calling json file.
aws resourcegroupstaggingapi tag-resources --cli-input-json file://tags.json
If you don't know exact format of json file then just run following, which will spit out format in tags.json file in current directory.
aws resourcegroupstaggingapi tag-resources --generate-cli-skeleton > tags.json
tags.json will have json. Just update the file and run the first commmand.
{
"ResourceARNList": [
""
],
"Tags": {
"KeyName": ""
}
}
You can fill up your data. e.g. for S3 bucket
{
"ResourceARNList": [
"arn:aws:s3:::my-s3-bucket"
],
"Tags": {
"Application": "My Application"
}
}
Trying to start a couple of spot instances within a simple script, and the syntax supplied in the AWS documentation and aws ec2 request-spot-instances help output is listed in either JAVA or JSON syntax. How does one enter the parameters under the JSON syntax from inside a shell script?
aws --version
aws-cli/1.2.6 Python/2.6.5 Linux/2.6.21.7-2.fc8xen
aws ec2 request-spot-instances help
-- at the start of "launch specification" it lists JSON syntax
--launch-specification (structure)
Specifies additional launch instance information.
JSON Syntax:
{
"ImageId": "string",
"KeyName": "string",
}, ....
"EbsOptimized": true|false,
"SecurityGroupIds": ["string", ...],
"SecurityGroups": ["string", ...]
}
I have tried every possible combination of the following, adding & moving brackets, quotes, changing options, etc, all to no avail. What would be the correct formatting of the variable $launch below to have this work? Other command variations -- "ec2-request-spot-instances" are not working in my environment, nor does it work if I try to substitute --spot-price with -p.
#!/bin/bash
launch="{"ImageId":"ami-a999999","InstanceType":"c1.medium"} "SecurityGroups":"launch-wizard-6""
echo $launch
aws ec2 request-spot-instances --spot-price 0.01 --instance-count 1 --type c1.small --launch-specification $launch
This provides result:
Unknown options: SecurityGroups:launch-wizard-6
Substituting the security group number has the same result.
aws ec2 describe-instances works perfectly, as does aws ec2 start-instance, so the environment and account information are properly setup, but I need to utilize spot pricing.
In fact, nothing is working as listed in this user documentation: http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-RequestSpotInstances.html
Thank you,
I know this is an old question, but in case somebody runs into it. I had the same issue recently with the CLI. It was very hard to get all the parameters to work correctly for request-spot-instances
#!/bin/bash
AWS_DEFAULT_OUTPUT="text"
UserData=$(base64 < userdata-current)
region="us-west-2"
price="0.03"
zone="us-west-2c"
aws ec2 request-spot-instances --region $region --spot-price $price --launch-specification "{ \"KeyName\": \"YourKey\", \"ImageId\": \"ami-3d50120d\" , \"UserData\": \"$UserData\", \"InstanceType\": \"r3.large\" , \"Placement\": {\"AvailabilityZone\": \"$zone\"}, \"IamInstanceProfile\": {\"Arn\": \"arn:aws:iam::YourAccount:YourProfile\"}, \"SecurityGroupIds\": [\"YourSecurityGroupId\"],\"SubnetId\": \"YourSubnectId\" }"
Basically what I had to do is put my user data in an external file, load it into the UserData variable and then pass that on the command line. Trying to get everything on the command line or using an external file for the ec2-request-spot-instances just kept failing. Note that other commands worked just fine, so this is specific to the ec2-request-spot-instances.
I detailed more about what i ended up doing here.
You have to use a list in this case:
"SecurityGroups": ["string", ...]
so
"SecurityGroups":"launch-wizard-6"
become
"SecurityGroups":["launch-wizard-6"]
Anyway, I'm dealing with the CLI right now and I found more useful to use a external JSON
Here is an example using Python:
myJson="file:///Users/xxx/Documents/Python/xxxxx/spotInstanceInformation.json"
x= subprocess.check_output(["/usr/local/bin/aws ec2 request-spot-instances --spot-price 0.2 --launch-specification "+myJson],shell=True)
print x
And the output is:
"SpotInstanceRequests": [
{
"Status": {
"UpdateTime": "2013-12-09T02:41:41.000Z",
"Code": "pending-evaluation",
"Message": "Your Spot request has been submitted for review, and is pending evaluation."
etc etc ....
Doc is here : http://docs.aws.amazon.com/cli/latest/reference/ec2/request-spot-instances.html
FYI - I'm appending file:/// because I'm using MAC. If you are launching your bash script using Linux, you could just use myJson="/path/to/file/"
The first problem, here, is quoting and formatting:
$ launch="{"ImageId":"ami-a999999","InstanceType":"c1.medium"} "SecurityGroups":"launch-wizard-6""
This isn't going to generate valid JSON, because the block you copied from the help file includes a spurious closing brace from a nested object that you didn't include, the closing brace is missing, and the unescaped double quotes are disappearing.
But we're not really getting to the point where the json is actually being validated, because with that space after the last brace, the cli is assuming that SecurityGroups and launch-wizard-6 are more command line options following the argument to --launch-specification:
$ echo $launch
{ImageId:ami-a999999,InstanceType:c1.medium} SecurityGroups:launch-wizard-6
That's probably not what you expected... so we'll fix the quoting so that it looks like one long argument, after the json is valid:
From the perspective of just generating valid json structures (not necessarily content), the data you are most likely trying to send would actually look like this, based on the docs:
{"ImageId":"ami-a999999","InstanceType":"c1.medium","SecurityGroups":["launch-wizard-6"]}
Check that as structurally valid JSON, here.
Fixing the bracing, commas, and bracketing, the CLI stops throwing that error, with this formatting:
$ launch='{"ImageId":"ami-a999999","InstanceType":"c1.medium","SecurityGroups":["launch-wizard-6"]}'
$ echo $launch
{"ImageId":"ami-a999999","InstanceType":"c1.medium","SecurityGroups":["launch-wizard-6"]}
That isn't to say the API might not subsequently reject the request due to something else incorrect or missing, but you were never actually getting to the point of sending anything to the API; this was failing local validation in the command line tools.