I'm trying to use the select command of AWS SimpleDB from AWS CLI.
The required call is as follows:
select
--select-expression <value>
with select-expression being described as follows: --select-expression (string) The expression used to query the domain.
The select is supposed to be similar to the SQL select statement, however I keep getting errors about the syntax, e.g.:
aws sdb select --select-expression "select * from my-domain"
An error occurred (InvalidQueryExpression) when calling the Select operation: The specified query expression syntax is not valid.
I can't find any documentation or example about the right syntax to use, either.
I found the solution - turns out I needed to use single quotes for the query and special character around the table name:
aws sdb select --select-expression 'select * from `my-domain`'
Related
So I'm trying to execute the following in AWS Athena which allows to run only one statement at a time:
MSCK REPAIR TABLE some_database.some_table_001;
MSCK REPAIR TABLE some_database.some_table_002;
MSCK REPAIR TABLE some_database.some_table_003;
Problem is, I just don't have three statements, I have 700+ similar statements and would like to run those all 700+ in one go as batch.
So using AWS CloudShell CLI and tried running the following:
aws athena start-query-execution --query-string "MSCK REPAIR TABLE `some_table_001`;" --work-group "primary" \
--query-execution-context Database=some_database \
--result-configuration "OutputLocation=s3://some_bucket/some_folder"
..hoping I could use Excel to generate 700+ statements like this and run as batch
..but keep getting this error:
An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: line 1:1: mismatched input 'MSCK'. Expecting: 'ALTER', 'ANALYZE', 'CALL', 'COMMIT', 'CREATE', 'DEALLOCATE', 'DELETE', 'DESC', 'DESCRIBE', 'DROP', 'EXECUTE', 'EXPLAIN', 'GRANT', 'INSERT', 'PREPARE', 'RESET', 'REVOKE', 'ROLLBACK', 'SET', 'SHOW', 'START', 'UNLOAD', 'UPDATE', 'USE', <query>
Not sure what I'm doing wrong as the same MSCK command seems to run fine in Athena console. I know Athena is finicky when it comes to
`some_table_001`
versus
'some_table_001'
(different types of single quotes), I tried both but didn't get it work.
Any thoughts on possible solution?
For Amazon Athena, special characters other than underscore (_) are not supported in the table names and table column names.
Since your table name doesn't contain special characters, you don't need to wrap it in quotes or backticks. However, had your table name contained special characters then you would have had to use backticks. Refer below Amazon Athena Documentation for more details here.
Regarding running multiple MSCK REPAIR TABLE statements in Amazon Athena, you may also use any SDK like boto3 (Python).
I use Amazon Redshift Serverless and Query editor(v2) and I'm having trouble with user permissions.
The following error occurred when importing data(.csv) in S3.
ERROR: Load into table 'x' failed. Check 'sys_load_error_detail' system table for details.
Therefore, I executed the command Select * From stl_load_errors to check the error, but it did not work well.
ERROR: permission denied for relation stl_load_errors
I checked my user permissions using select * from pg_user;and they are presented as follows.
However, I don't see any problem, what is the problem?
(I use hoge.)
usename
usesysid
usecreatedb
usesuper
usecatupd
passwd
valuntil
useconfig
rdsdb
1
true
true
true
********
infinity
NULL
hoge
101
true
true
false
********
NULL
NULL
I have tried to look at the query in the Amazon Redshift Serverless (Preview) dashboard under "Query and Database Monitoring", but could not find any details about the error.
What should I do?
AWS Redshift serverless is not exposing the stl_load_errors table
Perhaps you should try SYS_LOAD_HISTORY and SYS_LOAD_ERROR_DETAIL.
https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-monitoring.html
You can't query STL, STV, SVCS, SVL, and some SVV system tables and views with Amazon Redshift Serverless, except the following:
...see link above....
try
select * from sys_load_error_detail
I am trying to merge a bigquery table (target) with a google cloud postgres table (source) with a federated query. However it looks like bigquery will not accept a federated query in the "using" clause.
Syntax error: Expected "(" or keyword SELECT or keyword WITH but got identifier "EXTERNAL_QUERY" at [3:9]
My query looks something like the below.
MERGE bigquery_dataset.bigquery_table TARGET
USING (
EXTERNAL_QUERY("projects/company-co/locations/us/connections/company","SELECT * FROM postgres_schema.postgres_table")
) SOURCE
ON target.id = source.id
WHEN MATCHED THEN ...
WHEN NOT MATCHED BY TARGET THEN ...
WHEN NOT MATCHED BY SOURCE THEN ...
Are there any known workarounds for this type of functionality? Or is there any other way to perform this type of merge?
As per your requirement if you want to run federated queries in BigQuery where your external data source is located in Cloud PostgreSQL instance, you need to define the source dataset using the SQL function i.e EXTERNAL_QUERY
The error you are getting : “Syntax error: Expected "(" or keyword SELECT or keyword WITH but got identifier "EXTERNAL_QUERY" at [3:9]” is because you are missing out the SELECT statement before your EXTERNAL_QUERY.
As per this doc, the syntax should be :
SELECT * FROM EXTERNAL_QUERY(connection_id, external_database_query[, options]);
I tried running the federated query in BigQuery where the source is in Cloud PostgreSQL and it is working as expected.
SQL QUERY :
MERGE myproject.demo.tab1 TARGET
USING (
select * from EXTERNAL_QUERY("projects/myproject/locations/us-central1/connections/sqltobig", "SELECT * FROM entries;")
) SOURCE
ON target.entryID = source.entryID
WHEN MATCHED THEN
DELETE
WHEN NOT MATCHED THEN
INSERT(guestName, content, entryID)
VALUES(guestName, content, entryID)
I want to add manually a partiton to the datacalaog.
I can do it using Athena editor but i want to make it using a shell script that i can parameter and schedule.
Here's an example of my sql command to add a partition:
ALTER TABLE tab ADD PARTITION (year='2020', month='10', day='28', hour='23') LOCATION 's3://bucket/data/2020/10/28/23/'
i want to do the same thing with a shell command.
I think that i can use Glue APi: create-partition
Doc: https://docs.aws.amazon.com/cli/latest/reference/glue/create-partition.html
I'm trying but there something wrong with the format of the parameter that i add.
I'm trying to query my AWS Cloudsearch (2013 API) domain using the AWS CLI on Ubuntu. I haven't been able to get it to work successfully when the search is restricted to a specific field. The following query:
aws --profile myprofile cloudsearchdomain search
--endpoint-url "https://search-mydomain-abc123xyz.eu-west-1.cloudsearch.amazonaws.com"
--query-options {"fields":["my_field"]}
--query-parser "simple"
--return "my_field"
--search-query "foo bar"
...returns the following error:
An error occurred (SearchException) when calling the Search operation: q.options contains invalid javascript object
If I remove the --query-options parameter from the above query, then it works. From the AWS CLI docs regarding the fields options of the --query-options parameter:
An array of the fields to search when no fields are specified in a search... Valid for: simple , structured , lucene , and dismax
aws cli version:
aws-cli/1.11.150 Python/2.7.12 Linux/4.10.0-28-generic botocore/1.7.8
I think the documentation is a bit misleading as JSon does not like embedded double quotes inside double quotes, you would need to replace with single quote as
--query-options "{'fields':['my_field']}"
or you can escape the double quote
--query-options "{\"fields\":[\"my_field\"]}"