I have 10 tables under my dataset. I need to create "BigQuery Metadata Viewer" permission but would like to neglect 2 tables under my dataset. So that BigQuery Metadata Viewer policy only will be able to access 8 tables.
I see that there is "condition" tab but could not figure out how to apply such a condition here.
IAM condition is a nice way to solve that issue, but it's not available for BigQuery resources.
The solution here is to have 2 datasets
One with the 8 tables and the permission to view the metadata
one with the 2 other tables without the permission to view the metadata.
You can use the GRANT statement using the role bigquery.metadataViewer or dataviewer.You can set this role to table level, the user will have permission to a specific table, and won’t see listed tables. In this case, you need to know the name tables.
Take a look to this example:
GRANT `roles/bigquery.metadataViewer`
ON TABLE `my_dataset._my_table`
TO "user:user#domain.com"
Additionally, you can set this role at dataset level, this will grant access to read and list all the tables from the dataset.
Here’s an example:
GRANT `roles/bigquery.metadataViewer`
ON schema `project_name.dataset_name`
TO "user:mail#mail.com"
Related
I am trying to grant access to a table and cannot tell the difference between the "Viewer" and "BigQuery Data Viewer" roles. I do not want to give permissions to view other tables or datasets within the GCP Project or full access to BigQuery. Can someone tell me the difference between these two if the access is given through the Share option at the table level?
Screenshot of where Role is granted
At the moment I've assigned the user both roles and am not sure if I'm granting too much access. I want to follow the Principle of Least Privilege.
Viewer or roles/viewer is a top level, project level permission. Permissions for read-only actions that do not affect state, such as viewing (but not modifying) existing resources or data.
BigQuery Data Viewer its a resource based permission. Can be applied to BigQuery itself, or even to individual tables or views. Read all about its usage.
it has the following BQ roles:
bigquery.datasets.get
bigquery.datasets.getIamPolicy
bigquery.models.export
bigquery.models.getData
bigquery.models.getMetadata
bigquery.models.list
bigquery.routines.get
bigquery.routines.list
bigquery.tables.createSnapshot
bigquery.tables.export
bigquery.tables.get
bigquery.tables.getData
bigquery.tables.getIamPolicy
bigquery.tables.list
resourcemanager.projects.get
resourcemanager.projects.list
Problem: I have a project in BigQuery where all my data is stored. Within this project I created multiple datasets containing different views. Now I want to use different service accounts to query the different datasets containing different views via grafana (if that matters). These users should only be able to query the views (and therefore a specific dataset) meant for them.
What I tried: I granted BigQuery User, Viewer or Editor permissions (I tried all of them) at a dataset level (and also BigQuery Meatadata Viewer at a project level). When I query a view, I receive the error:
User does not have bigquery.jobs.create permission in project xy.
Questions: It is not clear to me if granting bigquery.jobs.create permission on project level, will allow the user to query all datasets instead of only the one I want him to access to.
Is there any way to allow the user to create jobs only on a single dataset?
Update October 2021
I've just seen that this question did go unanswered for me back then but still gets a lot of views. I believe the possibilities changed a bit since I asked the question so here is how I'm handling it now:
I give the respective service account the role roles/bigquery.jobUser on project level. This allows it to create jobs in general, however since I don't give any other permissions yet it cannot query data yet.
Then I give the role roles/bigquery.dataViewer on the dataset level. That makes it possible for the service account to query only the dataset I granted the permission on.
It is also possible to grant roles/bigquery.dataViewer on table level, what will restrict access to only the specific table.
In case you want the service account not only to query (view) the data, but also to insert or change it for example, replace roles/bigquery.dataViewer with the role having the necessary permissions (or assign that role in addition).
How to grant the permissions:
On dataset level
On table or view level
We had a same problem, how we solved was, created a custom role and assigned the custom role to the particular dataset.
You can grant bigquery.user role to a specific dataset as indicated in this guide. The bigquery.user role contains the bigquery.jobs.create permission as well as other basic permissions related to querying datasets. You can check the full list of permissions for this role in this list.
As suggested above, you can also create custom roles having only the exact permissions you want by following this piece of documentation.
How do I use athena workgroups to restrict access of a user to a particular database?
For e.g. I have a user "readonly" who should not be able to run select query on default database. Is this possible?
The way to restrict users from querying tables is to use IAM permissions. The permissions model in Athena is unfortunately more complicated than in an isolated data warehouse or RDBMS, since Athena is a part of a larger ecosystem that also includes S3 and Glue.
There is no specific permission for running SELECT. You can restrict users to run queries by controlling whether or not they are allowed to perform the athena:StartQueryExecution action, but you can't control what kind of queries they run.
Instead you need to think in terms of access to data, and access to the catalog.
To restrict reading you restrict the user's access to the data on S3. Even if a user is allowed to run a SELECT query they will get an error if they don't have permission to run s3:ListObject and s3:GetObject on the objects in the table's prefix.
You can also restrict a user's access to the catalog objects, i.e. the databases and tables – but that does not restrict their access to the data itself, think of it more as a restriction on creating, updating, and dropping databases and tables. Even if there is a way to restrict which databases and tables a user can see in the catalog, if they have permission to read the data they can read the data directly from S3 and skipping Athena.
You can find the documentation on how to control access to catalog objects here: https://docs.aws.amazon.com/athena/latest/ug/fine-grained-access-to-glue-resources.html
Workgroups in Athena can't be used to control access to data, nor to the catalog.
I have several customer projects that write analytic events into a BigQuery dataset. The setup is organised like this:
1) Each GCP project has its own set of GCP resources and some of them report analytics using BigQuery insert API.
2) There's a single "Main Analytics" project that intakes all the data from the different projects in a standardised table (all projects write in the same data format).
I've created a custom IAM role in "Main Analytics" with the required permissions to execute a row insert operation:
bigquery.datasets.get
bigquery.tables.get
bigquery.tables.updateData
For every customer project I've created a unique service account with the above role. This allows each resource in any project to authenticate and insert rows (but not create/delete tables).
Problem: What I really want to do is limit the service accounts to write only to a specific dataset that intakes all the data. The above IAM role allows the service account to list all datasets/tables in the "Main Analytics" project and to insert into them.
If I use dataset permissions - add the service account email as a user to the dataset ACL - then it would have to be WRITER dataset role which would allow the service account to create & delete tables in the dataset which is too broad.
Combining the IAM role with the dataset permissions results in a union so the wider WRITER permission take effect over the narrower IAM role.
Anyway I can configure roles/permissions to allow each service account to insert and only-insert to a specific dataset?
You can drop the bigquery.datasets.get permission from the custom IAM role so that they can’t list all the datasets, and then in the dataset's permissions give the READER role instead of WRITER to the user for that specific dataset.
I have a IAM user with Role: BigQuery Data Editor
In my data set I did Share dataset added the user with Can Edit privileges.
However when I'm running my script which access BigQuery I get error 403
When I add to my IAM user the Role BigQuery User The script works.
The scripts runs only SELECT query from a table in this data set.
I don't understand why I must grant BigQuery User for this to work.
According to the documentation https://cloud.google.com/bigquery/docs/access-control
Rationale: The dataEditor role extends bigquery.dataViewer by issuing
create, update, delete privileges for the tables within the dataset
roles/bigquery.dataViewer has bigquery.tables.getData which get table data
What am I doing wrong here?
Having access to the data and being able to retrieve it with a query are different things and that's where the confusion is coming from.
Per the documentation, roles/bigquery.dataEditor has the following permissions:
Read the dataset's metadata and to list tables in the dataset.
Create, update, get, and delete the dataset's tables.
This means that the user with this role has access and manipulation rights to the dataset's information and the tables in it. An example would be that a user with this role can see all the table information by navigating to it through the GCP console (schema, details and preview tabs) but when trying to run a query there, the following message will appear:
Access Denied: Project <PROJECT-ID>: The user <USER> does not have bigquery.jobs.create permission in project <PROJECT-ID>.
Now let's check the roles/bigquery.user permissions:
Permissions to run jobs, including queries, within the project.
The key element here is that the BigQuery User role can run jobs and the BigQuery DataEditor can't. BigQuery Jobs are the objects that manage the BigQuery tasks, this includes running queries.
With this information, it's clearer in the roles comparison matrix that for what you are trying to accomplish you'll need the BigQuery DataEditor role (Get table data/metadata) and the BigQuery User role (Create jobs/queries).