In the current usecase, I am using authorized views to handle row level security in the BigQuery dataset and Dataplex to handle the governance and permissions.
When a BQ Dataset is added as an asset, only the tables under it are getting listed as entities and the views are not listed. Is it possible to list the views as well in the entities so that it will be easier for discoverability and handle permissions at a more granular level?
I have tried my best in checking the documentation and playing around in the tool but could not display the BQ views as entities.
Have you tried using the search and under search, navigate to Data types. here you could select data set you want and view you want to see?
Related
We have a bunch of BigQuery datasets and for some reason we need to give out an authorized dataset access to a dataset that's not owned by us or included in our project but the main concern here is that I need to have control over who he gives access to view our datasets from that authorized dataset. Any method or a best practice for this type of problem?
Basically, we did it this way. Gave that other project's dataset the "Authorized dataset" access because they need to build their own views and then open these views to other customers they have. It's now able to view our tables and run queries against our datasets but the problem is that we have no control over who they give access to their dataset that they're using against ours and we need to figure out a way to control this.
I'm trying to create an exclusion filter in a Logs Router sink that will exclude buckets with specific labels from data access audits. I have a label called purpose in all my buckets. But while writing the LQL, the query doesn't seem to be able to find the custom label. Can someone help?
resource.type="gcs_bucket"
resource.labels.bucket_name="<bucket-nm>"
labels.purpose="test"
I have a table which requires a drive access scope to be queried. I was wondering if there is a way to create a view which wouldn't require this permission to be queried.
It's not possible as per documentation about data drive access
You will need access to data drive.
But as a workaround you can move that data into a dataset on bigquery that will work like an authorized view. As per definition:
Giving a view access to a dataset is also known as creating an authorized view in BigQuery. An authorized view lets you share query results with particular users and groups without giving them access to the underlying tables. You can also use the view's SQL query to restrict the columns (fields) the users are able to query.
Still, your users will need have access to the dataset that stores the view.
For your data analysts to query the view, they need to be granted the bigquery.dataViewer role on the dataset containing the view.
On that way it would be possible to query data which have access restrictions. Even on google documentation there is a guide you can use to produce such query named Create an authorized view.
Superset users want to add calculated fields to their datasets, but they run into an error:
Changing this dataset is forbidden
Users have the following roles:
Gamma
tables_modify: [can tables on Superset, can save on Datasource, can get on Datasource, can datasources on Superset, menu access on Tables, can fetch datasource metadata on Superset, can read on Dataset, can write on Dataset]
DB: [database access on [their TimeScale DB], datasource access on [the dataset they want to edit]]
sql_lab
I can't find the role missing for them to be able to edit their datasets. They have read and write permission on it, and are the ones creating the datasets in the first place.
I looked into the database properties but could not find any setting there either. Moreover with admin role I am able to edit datasets so I guess what's missing here is a user role, I just can't find which one.
As official documentation says, Gamma users are not able to alter data sources. Alpha users can add and alter data sources.
Additional information can be read further in that document:
If the user does not have the all_datasource_access permission
granted, the user will only be able to see Slices or explore the
data sources that are granted to them.
We have BigQuery instances with various datasets for each datasets we want to monitor the usage,
like Number of Queries per datasets, Queries fired for each datasets, Number of users accessing the datasets.
Is there any way in which we can monitor BigQuery usage?
You can see some metrics here:
https://console.cloud.google.com/monitoring/dashboards/resourceList/bigquery_dataset?project=**[YOUR_PROJECTID_GOES_HERE]**
Some more info here as well: https://cloud.google.com/bigquery/docs/monitoring
You can also enable BigQuery audit logs, and query the audit tables to get some insights https://cloud.google.com/bigquery/docs/reference/auditlogs.
Probably to monitor users, queries and other fine-grained monitoring you will only be able to do so using the audit logs
Most likely the best choice here is to simply query the job metadata directly in aggregate, through the relevant INFORMATION_SCHEMA views.
See https://cloud.google.com/bigquery/docs/information-schema-jobs for details about the job views, which includes some simple query examples at the end.
The jobs views do provide a list of referenced_tables, and you can identify the encapsulating data from them. You'll likely need to consider how you report on queries that reference multiple datasets, particularly if you are reporting on metrics like bytes scanned or resources utilized.