I'm using latest AWSRDSData client for running queries on my Aurora-MySQL database.
The schema and table charsets are utf8mb4 and collation is utf8mb4_turkish_ci;
When I insert rows via MySQL workbench, there's no problem with unicode characters but when using RDS Java SDK, unicode characters such as \u015F appear as ?.
I couldn't figure out how to set charset of data client by scanning sdk methods, help is appreciated.
thanks in advance
The problem is solved thanks to AWS support team.
Here is the point: even though the charset and collation are set on SQL Engine, the following parameters should be set in RDS DB parameter groups, in order to let rds data client understand the charset and collation for connection.
collation_server
collation_connection
character_set_server
information about parameter groups: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.how-it-works.html#aurora-serverless.parameter-groups
Related
I Know that query reuse feature was recently added in aws athena.
The aws webconsole confirmed this operation.
But I cant' use this operation in dbeaver.
I tried to changed jdbc url parameter like (jdbc:athena?param1=val1..) but It didn't working.
the parameter is
enableResultReuse=1
ageforResultReuse=60
has anyone solved this problem ??
reference) https://aws.amazon.com/ko/blogs/big-data/reduce-cost-and-improve-query-performance-with-amazon-athena-query-result-reuse/
Try adding both parameters to Athena Driver properties (in connection settings) in DBeaver:
I am using JDBC to connect to Athena for a specific Workgroup. But it is by default redirecting to the primary workgroup
Below is the code snippet
Properties info = new Properties();
info.put("user", "access-key");
info.put("password", "secrect-access-key");
info.put("WorkGroup","test");
info.put("schema", "testschema");
info.put("s3_staging_dir", "s3://bucket/athena/temp");
info.put("aws_credentials_provider_class","com.amazonaws.auth.DefaultAWSCredentialsProviderChain");
Class.forName("com.simba.athena.jdbc.Driver");
Connection connection = DriverManager.getConnection("jdbc:awsathena://athena.<region>.amazonaws.com:443/", info);
As you can see I am using "Workgroup" as the key for the properties. I also tried "workgroup", "work-group", "WorkGroup". It is not able to redirect to the specified Workgroup. Always going to the default one i.e primary workgroup.
Kindly help. Thanks
If you look at the release notes of Athena JDBC, the workgroup support is from v2.0.7.
If you jar is below this version, it will not work. Try to upgrade the library to 2.0.7 or above
You need to Override Client-Side Settings in workgroup.Enable below setting and rerun the query via JDBC.
Check this doc for more information.
I am creating a postgres database in aws cloud through CFT. I want to set log_line_prefix value in the postgres.conf file.I am unable to set the values by adding it to db parameter group.Is there any other way to set this?
log_line_prefix isn't supported in RDS, Here is the workaround mentioned in the official documentation.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.Badger
You can use a log analyzer such as pgbadger to analyze PostgreSQL logs. The pgbadger documentation states that the %l pattern (log line for session/process) should be a part of the prefix. However, if you provide the current rds log_line_prefix as a parameter to pgbadger it should still produce a report.
I am trying to connect Athena with Apache Zeppelin.I need to handle secret_key, Access_key, and Session_token. I am feeling hard to establish my connection with the Zeppelin JDBC interpreter.
I am following the steps as mentioned in this block,
If any one can help me out in establishing the connection with AWS Session token approach that would be helpful.
Thank You
The main docs for this are here:
https://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html
I found there are 2 driver versions, -1.1.0 and -1.0.1 . I could only get Zeppelin working with 1.1.0, and the links on that page don't go to that file, the only way to get it was using the aws s3 cp command
e.g.
aws s3 cp s3://athena-downloads/drivers/AthenaJDBC41-1.1.0.jar .
although I've given feedback on that page so it should be fixed soon.
Regarding the parameters, you use default.user and enter the Access_Key, default.password and enter the Secret_key. The default.driver should be com.amazonaws.athena.jdbc.AthenaDriver
The default.s3_staging_dir is actually the bucket where csv results are written so needs to match your athena settings.
There is no mention of where you might put a session token, however, you could always try putting it on the jdbc connection string ( which goes in default.url parameter value)
e.g.
jdbc:awsathena://athena.{REGION}.amazonaws.com:443?SessionToken=blahblahsomethingrealsessiontokengoeshere
but of course, replace {REGION} with the actual aws region and use your real session token.
Currently using Amazon's Aurora DB and in our Jira we are seeing some names with question marks (???) due to their characters.
After further looking into this it seems that I need to change the character_set_server from latin1 to utf8.
In Aurora DB I looked in the parameter groups and cluster group settings but I don't see character_set_server, or anything similar.
Thank you.
As Mark B said, you can easily change it on "Parameter groups" page,
Open your cluster group settings for edit and enter "character_set_ser" in the filter and you should see it.
For more information read the following page:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html
https://aws.amazon.com/premiumsupport/knowledge-center/rds-modify-parameter-group-values/