I'm working at a company that used to use AWS Athena and Quicksight to run sql queries and create dashboards, but now we have to use Apache Superset to do this.
While all users was using aws console, I could get Cloud Trail logs to send to the managers some reports of data consumption based on Athena workgroups or even idetify users that ran heavy queries, but since we migrate to Superset I lost the trace of user and workgroup because all queries are run using the same Athena connection...
Is this possible to pass the Athena workgroup based on user role (and maybe the username too) throught the conector?
I tried to find something on superset docs, but didn't find anything :(
Related
Since the cloud-sql-proxy already forces individual user authentication with the database through a users iam account, and allows specifying read / write permissions, it seems potentially pointless to also have an individual database accounts for each user as well.
For security, is it necessary to have a database user per dev when using cloud-sql-proxy, or is it fine to just have one database user, since they are already authenticated by the time they can enter a database user / password anyways. I'm not a server dev or a DBA, so I thought it best to ask.
In fact, you have 2 levels of permissions
Cloud IAM allows you to access to Cloud SQL product or not
Database user management allows to log into the db engine and to get the db engine permission (access to a specific schema, one schema per developer, on the same SQL instance for instance).
The hosted database engine are based on MySQL, PostgreSQL or SQL Server. All those databases have their legacy user authentication in place. You have to deal with.
We have multiple users that submit queries to AWS Athena concurrently. Is there anyway Athena CLI that allows us to find out the submitter, given a query execution id?
Based on the AWS Athena doc, it seems that this is not supported.
https://docs.aws.amazon.com/cli/latest/reference/athena/
Use Cloudtrail service instead.
From User guide:
Using CloudTrail, you can determine the request that was made to Athena, the
IP address from which the request was made, who made the request, when
it was made, and additional details.
More details here: https://docs.amazonaws.cn/en_us/athena/latest/ug/monitor-with-cloudtrail.html
I have a Power BI workbook that I have created in Desktop. It sources from a SQL Server database. I can access this database with account x. My Azure tenant admin has created a data source for this database in our gateway (within the Power BI service), and I have access to this gateway. The admin supplied account y in connecting to this data source. How does this work when I go to refresh the dataset that this workbook creates when I publish it to the service? That is, when I schedule a refresh on the dataset, will it dial into the SQL Server database using account y provided in the data source definition (virtually ignoring / dropping account x's credentials)?
Yep. That's exactly how it works. The automated refresh will use account 'Y.'
Data sources that have been deployed to some hosted location will almost always disregard the credentials used to create the dataset and instead use credentials that are specifically supplied for the refresh. These 'service' accounts will typically have different rules about password resets, have the lowest appropriate levels of access, and be under the prevue of system administrators rather than report authors. Its a very standard practice. It protects against misuse, error, loss of accounts, and segregates actual user activity from automated behaviors in the logs.
However, it is a little odd to me that your admin 'created the datasource' -- is that correct? Or did the admin just wire up the gateway to the datasource that was deployed when you published?
If you want to use a datasource that is already published, then you need to connect to that datasource from PowerBI desktop. Otherwise you'll be pushing out something new that has nothing to do with the resources that your admin created.
Suppose I am an AWS Superuser who has all AWS permissions.
I have configured AWS glue including the connection to a database using username and password.
I have stored the username and password in AWS secret manager, the glue’s ETL job script will connect database using this information and then run the ETL job.
ETL Data engineers do not have super user permission. But they know how to write the details of the ETL job script. And the script needs to retrieve the secret info first, which means engineers can write code to print out the password… and we have a lot of Data engineers…
My question is: what is the right strategy to control the password access of the secret manager?
1) Shall we allow ETL data engineers to update script to glue and run it? then they can see the password, or
2) Shall we only allow them to write ETL script, but let superuser to update the script to glue after reviewing the code? or
3) Do we have a way to separate the ETL job script code and get_password code?
Note, I know How to use IAM, tags to control secret manager. But my question is different.
I wanted to setup an RDS instance store data for reporting. I have scrips that run different rest calls against certain sites that require bulk admin privilege on the back end because they dump their rest call data into a csv and then do a bulk csv insert into Sql Server SE. In my local environment setting up a user for my scripts to use with bulk admin privileges was easy. However, I couldn't seem to figure out how to do it in RDS. I opened a ticket with Amazon and they suggested writing a policy for it. So I figured I would ask here if this is possible and possible alternatives? If bulk/system admin privileges are out of the question in RDS I guess I will just have to use an AWS EC2 instance with Sql Server set up on it.
Bulk insert is not possible with RDS. The data_file parameter of the BULK INSERT command must refer to a file accessible by the machine running SQL Server.
Also, RDS does not support the bulkadmin server role.
Supported SQL Server Roles and Permissions
Importing and Exporting SQL Server Data