I'm trying to get metadata about Glue partitions via the CLI. I've tried following the docs but I keep getting the following error:
An error occurred (EntityNotFoundException) when calling the GetPartition operation: Cannot find partition.
I know that my CLI is set up right because I can get partitions when I runaws glue get-partitions --database-name my_db --table-name my_table --max-items 10.
Here are the commands that I've tried. What am I doing wrong with the syntax?
aws glue get-partition --database-name my_db --table-name my_table --partition-values "year" "2022", "month" "04", "day" "29", "hour" "16"
aws glue get-partition --database-name my_db --table-name my_table --partition-values "year" 2022, "month" 04, "day" 29, "hour" 16
aws glue get-partition --database-name my_db --table-name my_table --partition-values year 2022, month 04, day 29, hour 16
I think you're close. --partition-values is just looking for the values (not names and values)
Try the following
aws glue get-partition --database-name my_db --table-name my_table --partition-values "2022" "04" "29" "16"
Related
I created a DynamoDB table and was running the following command in the CLI
aws dynamodb put-item \
--table-name NBATeams \
--item '{"Team": {"S": "Team Name"},"Title": {"S": "Suns"}}' \
--region us-east-1
but I keep getting "An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: Missing the key Team Name in the item" Not sure What am I missing since my partition key is City.
aws dynamodb put-item \
--table-name NBATeams \
--item '{"Team": {"S": "Team Name"},"Title": {"S": "Suns"}}' \
--region us-east-1
i believe problem is with your partition key, please double checked the name of your "PARTTION_KEY " make sure all attribute names, values, and keys are case sensitive.
When doing a put-item you must provide the exact keys for the table, which includes the partition key and sort key if one is defined.
If you still face issue, share the output of the describe-table command so we can understand your schema.
I believe your key is Team Name
aws dynamodb put-item \
--table-name NBATeams \
--item '{"Team Name": {"S": "Team Name"},"Title": {"S": "Suns"}}' \
--region us-east-1
I am trying to put a variable in the AWS CLI DynamoDB command for add a row in a table:
aws dynamodb put-item --table-name table_name --item '{"id_proceso": {"N": "1"}, "fecha_proceso": {"S": "20210707170486"}, "status": {"S": "OK"}, "fecha_termino": {"S": "fecha_termino_var" } }' --region us-east-1
Where fecha_termino_var it should be a variable. example: fecha_termino_var=(date +'%Y/%m/%d %H:%M:%S')
but I have tried various ways but I have not managed to do it, is there any way to enter a variable in a aws cli command line?
The goal it is to add a row, which will contain a variable of the date in real time.
I've tried various ways but it just adds the variable type, in this case "date", or just the variable name in text plain "fecha_termino_var".
I'm working in a shell (.sh) to do this.
Failed image adds:
Your screenshot shows that you are missng " infront of fecha_termino. So it should be:
, "fecha_termino":
not:
, fecha_termino":
Update:
The full correct command is:
aws dynamodb put-item --table-name mcp_control_proceso --item '{"id_proceso": {"N": "1"}, "fecha_proceso": {"S": "20210707170487"}, "status": {"S": "OK"}, "fecha_termino": {"S": "'"${fecha_termino_var}"'"} }' --region us-east-1
I am very new to aws. I was trying to create a database table by running following command:
``
aws dynamodb create-table --table-name pizza-order --attribute-definitions AttributeName=orderId, AttributeType=S --key-schema AttributeName=orderId,keyType=HASH --provisioned-throughput ReadCapacityUnit=1,WriteCapacityUnit=1 --region us-east-2 --query TableDescription.TableArn --output text
``
But I am getting a error like this:
Error parsing parameter '--attribute-definitions': Expected: '', received: '' for input:
AttributeName=orderId,
The command works with following modifications:
Remove space after orderId,
KeyType correctly capitalized
ReadCapacityUnits and WriteCapacityUnits correctly pluralized.
Here is the working command for your reference:
aws dynamodb create-table --table-name pizza-order --attribute-definitions AttributeName=orderId,AttributeType=S --key-schema AttributeName=orderId,KeyType=HASH --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 --region us-east-2 --query TableDescription.TableArn --output text
aws dynamodb create-table --table-name pizza-order --attribute-definitions AttributeName=orderId,AttributeType=S --key-schema AttributeName=orderId,KeyType=HASH --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 --region us-east-2 --query TableDescription.TableArn --output text
Docs for reference
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/getting-started-step-1.html
To verify that DynamoDB has finished creating the Music table, use the describe-table command.
aws dynamodb describe-table --table-name pizza-order | grep TableStatus
This command returns the following result. When DynamoDB finishes creating the table, the value of the TableStatus field is set to ACTIVE.
"TableStatus": "ACTIVE",
note:- whenever you are stuck or want to know the details of the command its a good practice to check aws cli doc for particular API for create table
I have data in S3 that is partitioned by category and date as follows:
s3://mybucket/category=1/date=2018-08-30/data1.json
s3://mybucket/category=1/date=2018-08-31/data2.json
s3://mybucket/category=2/date=2018-08-30/data3.json
s3://mybucket/category=2/date=2018-08-31/data4.json
After running the crawler, I have two partition keys in my metadata table: one for category, the other for date. I want to retrieve partitions that match certain keys using the GetPartitions API so I began experimenting with the AWS CLI. If I run this command:
aws glue get-partitions --database-name mydb --table-name mytable --expression "category=1" --region us-west-2
I successfully retrieve the partition as expected. However, I tried the following command:
aws glue get-partitions --database-name mydb --table-name mytable --expression "category=1 AND date=2018-08-30" --region us-west-2
and the response was
An error occurred (InvalidInputException) when calling the
GetPartitions operation: Unsupported expression '2018 - 08 - 30'
Another command that produced this error was
aws glue get-partitions --database-name mydb --table-name mytable --expression category=1\ AND\ date=2018-08-30 --region us-west-2
I also tried modifying the call by using the following command:
aws glue get-partitions --database-name mydb --table-name mytable --expression "category=1 AND date=2018\-08\-30" --region us-west-2
which gave me the error
An error occurred (InvalidInputException) when calling the GetPartitions operation: Lexical error at line 1, column 35. Encountered: "\" (92), after : ""
Is the GetPartitions API able to handle expressions for partitions that contain hyphens? If so, what is the correct syntax?
Partitions that are initially generated by a crawler in AWS Glue will have type String in the metadata catalog. While some of my categories contained hyphens, they were in uuids (i.e. category=so36-fkw1-...) so they were not interpreted as expressions. On the other hand, the dates contain only numeric characters and - which was the root of the problem. I was able to fix it by enclosing the dates in singular quotes as follows:
aws glue get-partitions --database-name mydb --table-name mytable --expression category=1\ AND\ date=\'2018-08-30\' --region us-west-2
I have a DynamoDB's Table called "ZombieSession" and the "SessionId" primary key with "S" type.
The local service is running in http://localhost:8181.
For local tests, I'm trying execute these commands:
(1)
aws dynamodb delete-item --table-name ZombieSession --key
'4ae40a08-007c-4785-babd-caff0ed12d1d' --endpoint-url
http://localhost:8181 --region us-east-1
That results in:
Error parsing parameter '--key': Invalid JSON:
'4ae40a08-007c-4785-babd-caff0ed12d1d'
and
(2)
aws dynamodb delete-item --table-name ZombieSession --key
'{"SessionId":{"S":"4ae40a08-007c-4785-babd-caff0ed12d1d"}}'
--endpoint-url http://localhost:8181 --region us-east-1
That results in:
Error parsing parameter '--key': Invalid JSON:
'{SessionId:{S:4ae40a08-007c-4785-babd-caff0ed12d1d}}'
I don't found any documentation example about this.
What's the appropriate command for this operation?
I discovered that the value of --key parameter need have the quotation mark with escape:
aws dynamodb delete-item --table-name ZombieSession --key
"{\"SessionId\":{\"S\":\"4ae40a08-007c-4785-babd-caff0ed12d1d\"}}"
--endpoint-url http://localhost:8181 --region us-east-1